Author: José Postiga
-
Recently I had to upgrade my local PostgreSQL version from 10.10 to 11.8, to mirror the same version that exists in the production environment. I originally thought the upgrade would be simple: turn off the container, update the docker-compose service to reference the new version, and then spin the container.
However, as soon as the container went up, an error started showing up in the logs:

The error is very explicit: my data, because it was created by an earlier version, was incompatible with the new version. When I searched for possible solutions to this problem, all I could read was related to MacOS, and relied on using Brew to execute some shell commands that would take care of upgrading the data and making it compatible with the new version.
The problem is that I use Docker for this (well, for everything web-related, really), and since I use Linux Ubuntu, I can’t use Brew (it’s only for MacOS). So I needed a solution that was quick, reliable, and that leveraged Docker (preferably) so that I don’t need to install things on my local machine.
After a lot of searching, and some discovery of my own, I came to the following, simple, three-step based solution.
Step 1: Make a backup of your current data
With the old PostgreSQL service still running, we’ll execute the following command on the directory where the docker-compose file is located:
docker-compose exec {service_name} pg_dumpall -U {postgres_user} > dump.sqlThis command makes a SQL file which will contain all our current data, from every database, roles, etc. We’ll use this later on when we import it to the new PostgreSQL version.
If you want to do it without relying on a docker-compose file configuration, you can spin up a container dedicated to this process with the following command:
docker run --rm -v ${pwd}:/tmp -v {local_postgres_data_folder}:/var/lib/postgresql/data -w /tmp {postgres_image}:{current_version_tag} pg_dumpall -U {postgres_user} > dump.sqlStep 2: Delete your current service mapped volume
We can’t use the old data structure because it’ll be incompatible with the new version’s. There’s no other option, here, but to remove the volume our old PostgreSQL service uses so that we can update the version tag, later, and let the service create the base folders/files as if it was a first clean run.
Let’s find where the volume is mapped to, open a terminal, go to that location, and run the command to delete it completely:
sudo rm -rf {postgres_volume_directory}Step 3: Update the PostgreSQL version
With a backup of our current data, we can now shutdown the old PostgreSQL service container, update the docker-compose configuration to reference the newer version tag, and spin that service back up.
After the service is done starting, it should be referencing a new volume with fresh-created files that are supported by the new version. We only need to kick-off the import process so that our backup can also be used by this new version. We can do it by using the internal bash terminal, with the following command:
# Connect to the container's bash terminal docker-compose exec {service_name} bash # Now we run the import command psql -U {postgres_user} -d {default_postgres_database} < {mapped_volume_folder_path}/dump.sqlNote: this command only works if you have a volume mapped inside the container, on your docker-compose file.
If you prefer a one-off command to run this import, you can use the following command:
docker run --rm -v ${pwd}:/tmp -v {local_postgres_data_folder}:/var/lib/postgresql/data -w /tmp {postgres_image}:{new_version_tag} psql -U {postgres_user} -d {default_postgres_database} < dump.sqlAnd that’s it! We now have a new version of PostgreSQL running and we kept all our data compatible with that version.
-
Schedule
- 18h30 – 18h40: Welcome introduction
- 18h40 – 19h10: Slot machines. No, this is not gambling. It’s leveraging – João Patrício, Sotware Engineer @ Grupo Valco
- 19h10 – 19h40: Using abstract resources in Laravel Nova – Bruno Falcão, Program Manager @ Roche
- 19h40 – 20h00: Q&A and Networking
-
-
-
-
Recently I needed to tap on Laravel’s Passport programming to control how the JSON Web Tokens (JWT) were being issued. Specifically, I needed to add more claims to it (to hold more user information) and to control how the scopes were being generated. The idea was to add information like the authenticated user’s email, VAT number, account type, and, also, to forcibly add the scopes that were associated with the user’s role.
However, changing Laravel Passport’s behavior to make it happen isn’t quite obvious, and in a world were service-oriented architectures are becoming ever more common, JWT being the de facto way of carrying user’s information through multiple services, and since I couldn’t find quite a good resource to understand how to do it, I thought I could share with you how I approached and solved my own problem.
The need for control
I’m developing ITsoup, an IT support/helpdesk software system, designed to simplify and streamline all its related processes, based in a service-oriented architecture. This means that I’ll have many services that handle specific domains of work. In order to authenticate the requests, properly, I needed some way of holding users’ information in a way that could be passed around from service to service. The de facto way of doing this is with JWTs, so I started investigating how I could make those changes to the JWT generation logic.
The problem with current Laravel Passport implementation is that it only includes one identification claim for the user: it’s internal ID. That’s not of much use for me on this project. Using only the ID would require any other services to request any additional user information to the Organization Domain service’s API if they need, for example, the e-mail.
Doing that way would create an unwanted, unnecessary, dependency between this service and all others that work with the user’s data and would increase the load on this Organization Domain service as new services and/or traffic increased.
Understanding Laravel Passport’s Service Provider
Before anything, I like to always understand what’s going on under the hood. By getting a broader context of the system I need to change, I can make informed decisions on how to approach a solution.
As far as I understood, it all goes down on
Laravel\Passport\PassportServiceProvider. Here we can find theregisterAuthorizationServer()method, which is responsible for defining how Laravel instantiates theAuthorizationServerclass and registers the supported oAuth2 authorization grants. But before that happens, there’s a call to amakeAuthorizationServer()method. This method is the one that’s responsible to define how theAuthorizationServergets instantiated, and injects the proper dependencies from the Laravel’s Container!The key class to this whole thing is the
Bridge\AccessTokenRepository, which is one of the dependencies injected. Since it’s being injected via the Container, we can take advantage of it and inject our own instance of that class, instead. This class has a very special method,getNewToken(), which is called when an enabled authorization grant needs to generate a new token.Hooking to the logic flow, here, and direct it through our own implementation of the
AccessTokenRepositoryclass enables us to control how the JWT is created, the information it holds, and many other aspects of it.Building bridges
So, now that I’ve pinpointed exactly the class that I need to override, I defined that a call to the
Bridge\AccessTokenRepositoryclass, within Laravel’s Container, would return an instance of my ownAccessTokenRepositoryclass, which would extend the previous one but with the single detail of overriding thegetNewToken()method. As of this moment, I successfully routed the logic to my own class.Now, this method needs to return an implementation of the
AccessTokenEntityInterface. This interface defines how to compute an access token (a JWT, for example). Laravel’s Passport implementation is exactly the one that The PHP League’s implements. In fact, not wanting to override that default behavior is one of the main reasons that Passport doesn’t supply any simple solution around this problem. In order for being able to add more claims, and control how JWT is generated – and even the scopes associated – we need to override this exact implementation.At this point, I just needed to return an implementation of the
AccessTokenEntityInterfacethat suited my needs. There’re two methods, specifically, that I needed to override:convertToJWT()andgetScopes(). The first is the one that actually generates the JWT, and the second one is the one that compiles the scopes to associate with that JWT.So, basically, my extended
convertToJWT()method looked like this:private function convertToJWT(CryptKey $privateKey): Token { return (new Builder()) ->permittedFor($this->getClient()->getIdentifier()) ->identifiedBy($this->getIdentifier()) ->issuedAt(\time()) ->canOnlyBeUsedAfter(\time()) ->expiresAt($this->getExpiryDateTime()->getTimestamp()) ->relatedTo((string) $this->getUserIdentifier()) ->withClaim('scopes', $this->getScopes()) ->withClaim('customer_id', $this->user->customer_id) ->withClaim('vat_number', $this->user->vat_number) ->withClaim('name', $this->user->name) ->withClaim('email', $this->user->email) ->withClaim('account_type', $this->user->account_type) ->getToken(new Sha256(), new Key($privateKey->getKeyPath(), $privateKey->getPassPhrase())); }And my
getScopes()method looked like this:public function getScopes(): array { return $this->user ->roles() ->pluck('scopes') ->flatten() ->unique() ->map(static function ($scope) { return new Scope($scope); }) ->toArray(); }Now, the JWT is generated with what I defined to be the relevant user information, and can be passed around and, hopefully, reduce the need to query the Organization Domain service if the JWT itself already holds the data needed.
-
We’ve recently launched Place Checkup, a web application that allows users to check if a given place follows the WHO recommended safety and prevention measures against COVID-19. Business owners, which are not yet Infraspeak‘s customers, can register through this website, claim their places and register how and what measures they’re actually following. In the end, they get an A, B, or C badge qualifying their places.
Since Infraspeak’s core software business is directly related to infrastructure maintenance management, and since we develop with an API-first approach, we already had the core functionalities available to easily use on Place Checkup. Besides that, and to avoid having multiple accounts spread over different projects, we defined that we’d leverage our existent API authentication, and centralize all the accounts there.
Our API uses Passport, an oAuth2 server implementation for Laravel projects, so by sending a user’s credentials to the API service, if they are valid, it returns an encoded JWT with the user information we’d need. To keep things simple, and to avoid having to implement a Machine-to-Machine authentication, we wanted to simply store the generated JWT in session, on Place Checkup, for further usage on subsequent calls to the API.
Laravel has a very good, and extendable, authentication system, referenced throughout the documentation as Guards. Their job is to know how an authentication request gets processed and how to communicate with a Provider, whose responsibility is to know how to get the requested data from whatever persistent layer it handles.
If correctly implemented, a custom Guard would allow us to tap into middlewares, facades, and helper methods commonly used within Laravel to handle authentication, like the
Authfacade, or getting the currently authenticated user via$request->user(), or even use theauthmiddleware to secure access to private routes. That looked exactly what we needed for this project!But adding a custom Guard and Provider proved to be a little more difficult than what the documentation shows to be. Reading through the sections that describe how Guards and Providers work, I could only see how to register the custom classes to the Container, which interfaces to implement, and which type of classes it needs to return. I couldn’t quite get a clear picture of how the flow was, so I realized I’d need to dive deep into the authentication system and try to map it from within.
Understanding Guards
Laravel’s documentation describes Guards as the following:
Guards define how users are authenticated for each request.
But, what does that really mean? That was the question I had on my head while deep on the
SessionGuardclass – a preexistent Guard shipped by default with the framework – trying to understand what’s going on under the hood. I figured that a Guard has the sole responsibility of receiving requests to authenticate a user and to call the configured Provider to fetch that user on the persistent layer.This means that, for example, when we call
Auth::attempt()to attempt a login action, Laravel will defer that call to the same method on the configured Guard. This Guard will, in turn, prepare all it needs to send that request through the related Provider, which may (or may not) return an instance of theAuthenticatableinterface, the object that represents the authenticated user.I’ve also found that we actually have two interfaces for Guards:
- The
Guardinterface, referenced in the docs, aimed to a stateless authentication, like APIs; - The
StatefulGuardinterface, that extends the previousGuardinterface but also defines how to persist the authentication information in the configured session storage.
The fundamental difference between them: state management. The knowledge of the existence of this
StatefulGuardinterface pointed me to a good direction of how I could develop a Guard that would persist the JWT in the session that integrates perfectly with the expectations of Laravel itself. I was finally in a good place on understanding the key concepts of the whole authentication system.Understanding Providers
Laravel’s documentation describes Providers as the following:
Providers define how users are retrieved from your persistent storage.
Looking at the two default providers that Laravel supports,
EloquentUserProviderandDatabaseUserProvider, I noticed that first returns an instance ofModel(an Eloquent model class) and the other handles direct calls to a database table and returns an instance ofGenericUser(a slight variation of a Value Object). Both follow the sameUserProviderinterface.I concluded that Providers are an implementation of the Repository pattern, and exist only to abstract away how you validate the user’s credentials and fetch their data on whatever persistent layer it’s stored in.
Actually, that’s probably why the user’s credentials are passed on inside an array structure: because Laravel doesn’t want to be opinionated towards how you authenticate your users. You can use a combination of e-mail and password, or a one-time hash (like magic links sent to an e-mail), or whatever you want to use. It’s pretty clever, actually!
But it’s basically this… It’s inside a Provider that you actually define what data you need to authenticate a user (the credentials) and how you communicate with the persistent layer to get the relevant data. No matter how that’s done, a Provider must return an object that implements the
Authenticatableinterface. This is to ensure that Laravel uses the correct methods when working with that object, directly, like when callingAuth::id().Making all parts work together nicely
I’m pretty sure that at this point you already have a clearer understanding of the role Guards and Providers take within the Laravel’s authentication workflow: Guards receive the request to authenticate a user and pass the credentials to the defined Provider. The Provider, in turn, queries the underlying persistent storage about the relevant user data associated with those credentials and returns it all the way back to the Guard, which makes that information available for usage.
But even if you create all the required classes, you still need to register them within Laravel’s Container, so that it knows how to call those classes:
- To register the Provider, we use the
Auth::provider()method and define how the framework should instantiate the custom provider class; - To register the Guard, we use the
Auth::extend()method and define how the framework should instantiate the custom Guard.
Let me show you exactly what we’ve done to register the custom Guard and Provider used on the Place Checkup project:
public function boot(): void { Auth::provider('infraspeak-users', static function (Application $app, array $config) { return new InfraspeakUserProvider($app->make(InfraspeakApiService::class), $config); }); Auth::extend('infraspeak-jwt', static function (Application $app, $name, array $config) { return new JwtSessionGuard( Auth::createUserProvider($config['provider']), $app->make('session.store') ); }); }The
InfraspeakUserProviderclass has all the code to communicate with our API service and validate a user’s credentials and returns an instance of theAuthenticatableinterface. TheJwtSessionGuardclass has the code that takes that object, from the Provider, and persists it to the session, avoiding new calls being dispatched to the API service while that session is active and valid, protecting our API service from being flooded with authentication requests every time a protected endpoint is being processed.The very final step in all this work was to update the
config/auth.php, and switch the default Guard and Provider used for the web routes:'guards' => [ 'web' => [ 'driver' => 'infraspeak-jwt', 'provider' => 'infraspeak-users', ], ],Now we could use Laravel’s native classes to authenticate a user and use our API service to serve as the data repository to get that information.
- The
-
Schedule
- 18h30 – 18h40: Welcome introduction
- 18h40 – 19h10: Testing back and testing forward – António Fernandes, Senior Backend Developer @ Infraspeak
- 19h10 – 19h40: Integrating Laravel 7 with Telegram and WhatsApp – Vinícius Melo, Backend Developer @ VC Cloud
- 19h40 – 20h00: Q&A and Networking