Track successful email deliveries, clicks and opens in your Laravel PHP application with Mailgun.
Published on Monday September 9th, 2019
Table Of Contents
- Get Laravel application set up up
- Get your mailgun account set up
- Database schema
- Mailgun webhook Controller
- Setting up the welcome email
- Handling the incoming webhook
Source if you want to download the final project
While building ContestKit there was a feature I wanted to allow users to know if the emails that were sent to the winners were delivered successfully. Thankfully this feature is relatively easy to add because of Mailgun's amazing API. Let's create a new Laravel application and get started.
Set up your Laravel Application
To get this process started I always use the Laravel installer to start with a fresh install.
(I will be using 6.0.1 which is the most recent at the time of this writing)
This will get a Laravel application installed, generate your .env file and install your composer dependencies. Once that is installed we’re going to install the Laravel UI package to quickly scaffold the auth views for our app. This will generate our login and registration views.
composer require laravel/uiOnce that is installed run
php artisan ui vue --authSet up Mailgun
Next, you’ll need a Mailgun account. If you don’t have one, you can sign up for a free account here that gives you 10,000 free email sends a month. Once you have created your account. Once you have your account all set up, head over here to get your API information. You’re going to need your Private API key and the domain. Once you have that, open your .env and let’s add some information so our application knows to use Mailgun as our mail provider.
MAIL_DRIVER=mailgunMAILGUN_DOMAIN=yoursendingdomain.comMAILGUN_SECRET=private-api-keySince we're going to be using Mailgun as our email driver, we need to make sure we satisfy all the driver prerequisites, in this case, we need to install Guzzle Http so run
composer require guzzlehttp/guzzle.We’re going to need to come back to Mailgun, later on, to set up some webhooks, but until then let's get our application sending some emails, creating database records corresponding to the emails sent and get the controllers to handle the incoming webhook information from Mailgun.
Set up our database
Next, let’s get our email message model set up. In your console, run the following command.
$ php artisan make:model Message -mThis command will make a model for you in App/Model call Message.php and it will also make a migration for a table called messages. Let’s open our migration and get our DB schema in. The migration should be called ${datatime-stamp}_create_messages_table.php.
<?php use Illuminate\Support\Facades\Schema;use Illuminate\Database\Schema\Blueprint;use Illuminate\Database\Migrations\Migration; class CreateMessagesTable extends Migration{ /** * Run the migrations. * * @return void */ public function up() { Schema::create('messages', function (Blueprint $table) { $table->bigIncrements('id'); $table->string('message_id')->index(); $table->morphs('sender'); $table->dateTime('delivered_at')->nullable(); $table->dateTime('failed_at')->nullable(); $table->integer('opened')->default(0); $table->integer('clicked')->default(0); $table->timestamps(); }); } /** * Reverse the migrations. * * @return void */ public function down() { Schema::dropIfExists('messages'); }}What this table is going to do is track the outbound email message for us. We’re going to use a sender morphTo relationship so that we can attach outbound messages to any model in our app that we want. This will translate into being able to attach the welcome email we send to the user we sent it to.
The $table->morphs('sender'); will create two columns for us, a sender_type which will be represented by the model that sent the message, the second will be the sender_id which will represent the id of the Model.
From there we will store the unique id generated for the email message. Mailgun will send this message id when it sends webhook events. We ask Mailgun to notify us of four different events.
- The successful delivery of a message.
- Any clicks on links in the email (if there are any links).
- Every time the email is opened for that email.
- Any permanently failed deliveries.
This failed delivery event is triggered when Mailgun attempts to send the email several but the receiving email service does not accept it. This usually results from a fake/bad email or your email was blocked by the mail service you were sending it to.
Next, let's add the message relationship to our App\User model. Open up that and add the following.
<?php namespace App; use App\Message;use Illuminate\Contracts\Auth\MustVerifyEmail;use Illuminate\Foundation\Auth\User as Authenticatable;use Illuminate\Notifications\Notifiable; class User extends Authenticatable{ use Notifiable; /** * The attributes that are mass assignable. * * @var array */ protected $fillable = [ 'name', 'email', 'password', ]; /** * The attributes that should be hidden for arrays. * * @var array */ protected $hidden = [ 'password', 'remember_token', ]; /** * The attributes that should be cast to native types. * * @var array */ protected $casts = [ 'email_verified_at' => 'datetime', ]; public function message() { return $this->morphOne(Message::class, 'sender'); }}This will allow us to attach the outbound message on the user.
Next, let's define the inverse relationship on the App\Message model.
<?php namespace App; use Illuminate\Database\Eloquent\Model; class Message extends Model{ protected $guarded = ['id']; protected $dates = [ 'delivered_at', 'failed_at', ]; public function sender() { return $this->morphTo(); }}Next, there is a great little tip/trick that you can do to help keep your database records for this sender_type column clean. Currently, if you create a record in your database using this relationship you'll end up with a DB record that looks like this.
| id | message_id | sender_type | sender_id | delivered_at | failed_at | opened | clicked | created_at | updated_at ||----|--------------------------------------------------|-------------|-----------|--------------|-----------|--------|---------|---------------------|---------------------|You can see that the sender_type is the namespace of your User model. This is fine if you never change the namespace of your model, to clean this up, Laravel has a great Relation::morphMap([...]); method to help make the above a little more readable and a lot less coupled to our namespace.
open your AppServiceProvider.php file and in the register() method add the following.
<?php namespace App\Providers; use Illuminate\Database\Eloquent\Relations\Relation;use Illuminate\Support\ServiceProvider; class AppServiceProvider extends ServiceProvider{ /** * Register any application services. * * @return void */ public function register() { Relation::morphMap([ 'users' => 'App\User', ]); } /** * Bootstrap any application services. * * @return void */ public function boot() { // }}By doing the above, when you create a message now, your database record will look like this.
| id | message_id | sender_type | sender_id | delivered_at | failed_at | opened | clicked | created_at | updated_at ||----|--------------------------------------------------|-------------|-----------|--------------|-----------|--------|---------|---------------------|---------------------|Isn't that SO MUCH better? If you want to read more about this, check out this post by Joseph Silber.
Set up our Mailgun webhook controller
All right, we're ready to move on now. Now let's make a controller for the webhooks. For this I like to namespace things to keep them organized. In your console enter the following.$ php artisan make:controller Webhooks/MailgunWebhookControllerNext, let's make a middleware for this controller.
$ php artisan make:middleware MailgunWebhookMiddlewareext, in your routes folder let's make a webhooks.php file. In here we’re going to add all our webhook routes. I do this to keep things organized and because we want this URL to not have any of the web middleware stack. I also don’t want to have my webhook URL for Mailgun to be prefixed by /api otherwise you could put it in there as well.
In your newly created webhooks.php file add the following route.
Route::group(['namespace' => 'Webhooks',], function () { Route::post('mailgun', 'MailgunWebhookController');});If you were to run php artisan route:list you’ll notice the webhook URL you just added doesn’t come up, we’ll need to update our RouteServiceProvider.php to include the routes in this file. So let’s open that up. In the map() function let's add a method called mapWebhookRoutes()
<?php namespace App\Providers; use Illuminate\Foundation\Support\Providers\RouteServiceProvider as ServiceProvider;use Illuminate\Support\Facades\Route; class RouteServiceProvider extends ServiceProvider{ /** * This namespace is applied to your controller routes. * * In addition, it is set as the URL generator's root namespace. * * @var string */ protected $namespace = 'App\Http\Controllers'; /** * Define your route model bindings, pattern filters, etc. * * @return void */ public function boot() { // parent::boot(); } /** * Define the routes for the application. * * @return void */ public function map() { $this->mapApiRoutes(); $this->mapWebRoutes(); $this->mapWebhookRoutes(); } /** * Define the "web" routes for the application. * * These routes all receive session state, CSRF protection, etc. * * @return void */ protected function mapWebRoutes() { Route::middleware('web') ->namespace($this->namespace) ->group(base_path('routes/web.php')); } /** * Define the "api" routes for the application. * * These routes are typically stateless. * * @return void */ protected function mapApiRoutes() { Route::prefix('api') ->middleware('api') ->namespace($this->namespace) ->group(base_path('routes/api.php')); } /** * Define the "webhook" routes for the application. * * These routes are typically stateless. * * @return void */ protected function mapWebhookRoutes() { Route::prefix('webhooks') ->namespace($this->namespace) ->group(base_path('routes/webhooks.php')); }}Okay now that we have our route, let's get our controller set up. Open your MailgunWebhookController.php and let's add an __invoke() method.
<?php namespace App\Http\Controllers\Webhooks; use App\Http\Controllers\Controller; class MailgunWebhookController extends Controller{ public function __invoke() { // we'll handel our inbound webhook here. }}Once you’ve done that let's test to see if everything is working. if you run php artisan route:list you should see an output like this.
+--------+----------+----------------------------+------------------+------------------------------------------------------------------------+-------------------------------------------------+| Domain | Method | URI | Name | Action | Middleware |+--------+----------+----------------------------+------------------+------------------------------------------------------------------------+-------------------------------------------------+| | GET|HEAD | / | | Closure | web || | POST | _ignition/execute-solution | | Facade\Ignition\Http\Controllers\ExecuteSolutionController | Facade\Ignition\Http\Middleware\IgnitionEnabled || | GET|HEAD | _ignition/health-check | | Facade\Ignition\Http\Controllers\HealthCheckController | Facade\Ignition\Http\Middleware\IgnitionEnabled || | GET|HEAD | _ignition/scripts/{script} | | Facade\Ignition\Http\Controllers\ScriptController | Facade\Ignition\Http\Middleware\IgnitionEnabled || | POST | _ignition/share-report | | Facade\Ignition\Http\Controllers\ShareReportController | Facade\Ignition\Http\Middleware\IgnitionEnabled || | GET|HEAD | _ignition/styles/{style} | | Facade\Ignition\Http\Controllers\StyleController | Facade\Ignition\Http\Middleware\IgnitionEnabled || | GET|HEAD | api/user | | Closure | api,auth:api || | GET|HEAD | home | home | App\Http\Controllers\HomeController@index | web,auth || | GET|HEAD | login | login | App\Http\Controllers\Auth\LoginController@showLoginForm | web,guest || | POST | login | | App\Http\Controllers\Auth\LoginController@login | web,guest || | POST | logout | logout | App\Http\Controllers\Auth\LoginController@logout | web || | POST | password/email | password.email | App\Http\Controllers\Auth\ForgotPasswordController@sendResetLinkEmail | web,guest || | GET|HEAD | password/reset | password.request | App\Http\Controllers\Auth\ForgotPasswordController@showLinkRequestForm | web,guest || | POST | password/reset | password.update | App\Http\Controllers\Auth\ResetPasswordController@reset | web,guest || | GET|HEAD | password/reset/{token} | password.reset | App\Http\Controllers\Auth\ResetPasswordController@showResetForm | web,guest || | POST | register | | App\Http\Controllers\Auth\RegisterController@register | web,guest || | GET|HEAD | register | register | App\Http\Controllers\Auth\RegisterController@showRegistrationForm | web,guest || | POST | webhooks/mailgun | | App\Http\Controllers\Webhooks\MailgunWebhookController | |+--------+----------+----------------------------+------------------+------------------------------------------------------------------------+-------------------------------------------------+Tip if you want to filter the results to a particular route pattern you can run this php artisan route:list --path=webhooks which will return any routes that match that pattern.
+--------+--------+------------------+------+--------------------------------------------------------+------------+| Domain | Method | URI | Name | Action | Middleware |+--------+--------+------------------+------+--------------------------------------------------------+------------+| | POST | webhooks/mailgun | | App\Http\Controllers\Webhooks\MailgunWebhookController | |+--------+--------+------------------+------+--------------------------------------------------------+------------+Okay, If you got that then you’re golden. Next step let's get an email that we can send. Let’s make a welcome email for users that register for our app. Since Laravel makes it dead simple to add auth login screens. In your console let's run these commands. The first will create the auth boilerplate for us which will give us the login and registration pages in our app. Next, we will make a Mailable that we can send our users when they register, finally a listener where we will send that welcome email from.
The Welcome email
$ php artisan make:auth$ php artisan make:mail WelcomeEmail --markdown=emails.welcome$ php artisan make:listener SendWelcomeEmailOnce you’ve run that and you visit your app in the browser, you’ll notice that you now have a login and register nav item. Pretty awesome!
Okay now open your EventServiceProvider.php file, in there you’ll notice that there are already events mapped out. let's add our handler to that Registered event.
<?php namespace App\Providers; use App\Listeners\SendWelcomeEmail;use Illuminate\Auth\Events\Registered;use Illuminate\Auth\Listeners\SendEmailVerificationNotification;use Illuminate\Foundation\Support\Providers\EventServiceProvider as ServiceProvider;use Illuminate\Support\Facades\Event; class EventServiceProvider extends ServiceProvider{ /** * The event listener mappings for the application. * * @var array */ protected $listen = [ Registered::class => [ SendWelcomeEmail::class, SendEmailVerificationNotification::class, ], ]; /** * Register any events for your application. * * @return void */ public function boot() { parent::boot(); // }}Okay now that you have that done, every time someone registers in your app, this handler will be triggered. If you’re wondering what the SendEmailVerificationNotification class does, it's used to send the verification email if you choose to require that in your application. You can learn more about that from here. let's start sending emails to welcome our users! in the SendWelcomeEmail class add the following.
<?php namespace App\Listeners; use App\Mail\WelcomeEmail;use Illuminate\Auth\Events\Registered;use Illuminate\Contracts\Queue\ShouldQueue;use Illuminate\Queue\InteractsWithQueue;use Illuminate\Support\Facades\Mail; class SendWelcomeEmail{ /** * Handle the event. * * @param object $event * @return void */ public function handle(Registered $event) { Mail::to($event->user)->send(new WelcomeEmail($event->user)); }}So now, if you go ahead and register you should get an email sent to you that looks like this.
This is the default markdown email that ships with Laravel if you see this, things are good. Now that we're here, let's start tracking the and saving the outbound messages. To do this we'll open our App/Mail/WelcomeEmail.php
We're going to want to add the following.
<?php namespace App\Mail; use App\User;use Illuminate\Bus\Queueable;use Illuminate\Contracts\Queue\ShouldQueue;use Illuminate\Mail\Mailable;use Illuminate\Queue\SerializesModels; class WelcomeEmail extends Mailable{ use Queueable, SerializesModels; /** * Create a new message instance. * * @return void */ public function __construct(User $user) { $this->user = $user; } /** * Build the message. * * @return $this */ public function build() { $this->withSwiftMessage(function ($message) { $this->user->message()->create([ 'message_id' => $message->getId(), ]); }); return $this->markdown('emails.welcome'); }}In the constructor of this class, we're going to get passed the User that we're sending this email to, we will be using this to store the outbound Message.
The build() code is where we hook into the SwiftMailer to get the message-id and using the message relationship we create the message record in our database. So let's try and register again and see what happens.
Hopefully, you should see a record created in the messages table linked to the user you just registered? If so let's move on to the webhooks handling.
Handling the incoming webhook
For this, we'll need to head back into our Mailgun account. From the dashboard select the sending domain that you configured earlier on and then select the webhooks from the sidebar.
I am running Laravel Valet locally so I'm going to run valet share and get a URL for my local app to use.
Once you're done adding the 4 events we will be tracking, and yes there are more if you want to track more you'll be able to add them with ease. So now that we have this let's start updating our messages. The first thing we're going to do is flesh out our middleware for our webhook controller. This middleware will ensure that the inbound HTTP request is in fact from Mailgun by verifying the webhook signature using our API key. Mailgun does refer to this as a webhook secret but if you look, you'll notice that it's the same token/string as your API key.
<?php namespace App\Http\Middleware; use Closure;use Illuminate\Http\Response;use Illuminate\Support\Facades\Log; /** * Validate Mailgun Webhooks * @see https://documentation.mailgun.com/user_manual.html#securing-webhooks */class MailgunWebhook{ /** * Handle an incoming request. * * @param \Illuminate\Http\Request $request * @param \Closure $next * @return mixed */ public function handle($request, Closure $next) { if (!$request->isMethod('post')) { abort(Response::HTTP_FORBIDDEN, 'Only POST requests are allowed.'); } if ($this->verify($request)) { return $next($request); } abort(Response::HTTP_FORBIDDEN); } /** * Build the signature from POST data * * @see https://documentation.mailgun.com/user_manual.html#securing-webhooks * @param $request The request object * @return string */ private function buildSignature($request) { return hash_hmac( 'sha256', sprintf('%s%s', $request->input('timestamp'), $request->input('token')), config('services.mailgun.secret') ); } public function verify($request) { $token = $request->input('signature.token'); $timestamp = $request->input('signature.timestamp'); $signature = $request->input('signature.signature'); if (abs(time() - $timestamp) > 15) { return false; } return hash_hmac('sha256', $timestamp . $token, config('services.mailgun.secret')) === $signature; }}In the above, we take the inbound request and analyze it to ensure it's authentic before allowing our controller to handle it. You can read more in the Mailgun docs. Below is an example of the payload you can expect from this webhook.
{ “signature”: { "timestamp": "1529006854", "token": "a8ce0edb2dd8301dee6c2405235584e45aa91d1e9f979f3de0", "signature": "d2271d12299f6592d9d44cd9d250f0704e4674c30d79d07c47a66f95ce71cf55" } “event-data”: { "event": "opened", "timestamp": 1529006854.329574, "id": "DACSsAdVSeGpLid7TN03WA", // ... }, "message": { "headers": { } }}In the verify method you can see the three main things we have here, the timestamp, the token and the signature. What we need to do is concatenate the timestamp and token values, encode the resulting string with the HMAC algorithm (using your API Key as a key and SHA256 digest mode). Then we compare the result to the signature we in get the webhook payload. We also check if the timestamp is not too old which would make it stale. If everything looks good from here, we let this request through and we handle it in our controller.
<?php namespace App\Http\Controllers\Webhooks; use App\Http\Controllers\Controller;use App\Http\Middleware\Webhooks\MailgunWebhook;use App\Message;use Illuminate\Http\Request; class MailgunWebhookController extends Controller{ /** * Constructor. */ public function __construct() { $this->middleware(MailgunWebhook::class); } /** * Handles the Stripe Webhook call. * * @param \Illuminate\Http\Request $request * * @return mixed */ public function __invoke(Request $request) { $data = $request->get('event-data'); $message_id = $data['message']['headers']['message-id']; if ($email = Message::whereMessageId($message_id)->first()) { if ($data['event'] === 'opened' || $data['event'] === 'clicked') { $email->increment($data['event']); } if ($data['event'] === 'delivered' || $data['event'] === 'failed') { $email->update(["{$data['event']}_at" => now()]); } } }}In the construct method, we define the middleware we want to run to protect this controller, and in the __invoke() method we wrote before we will extract the data we need from the request. The message-id that we set in our messages table is found in message -> headers -> message-id this is id to retrieve the message record we want to update. Next we can find the event type (ie, clicked, opened, delivered, failed) in the event-data -> event, so now all we need to do is add some logic for opened or clicked events we're going to increment a column, otherwise its a delivered or failed message so we'll update those timestamps to the current UTC of the event. There you have it.
And there you have it, from here you can add messages to other models in your app, say invoices, or any other event that triggers an email and track if your user got it, opened it or clicked etc.
I hope you have found this useful! If you have any comments or things you think can improve this, send me a message or Tweet or something.. Bonus points if it's a Drake gif.
