Published on June 05, 2022

Creating a newsletter subscription service with react and aws

27 minutes

What is the easiest way of creating a back-end service for a newsletter ? For me personally what comes to mind first based on my experience is serverless?, namely cloud functions and database cloud services. Certainly there are several options out there but this time around I'll choose those I'm more familiar with even though I don't have much experience on the topic, but yet what I'm about to walk you through is the approach I took in order to implement a newsletter on this blog.

I'm talking about aws lambda functions, dynamodb and api gateway

So let us summarize what are these three strangers I've just mentioned above beforehand shall we ?

What is aws lambda ?

Aws lambda is amazon's functions as a service's platform. It basically allows you to create a function that runs code upon certain events and provides you with a url so that you can use it as an endpoint from your mobile or web application sparing you from the burden of creating and mantaining a back-end service by yourself. You can even associate them to routes of the api gateway service.

It is ideal for this situation because:

It self auto-scales and auto-provisions, dependent on load.
It has costs that are based on precise usage, up from and down to zero usage.
We are lazy, frankly speaking.

What is dynamodb ?

Dynamodb is an amazon web services database system that supports data structures and key-valued cloud services. It allows users the benefit of auto-scaling, in-memory caching, backup and restore options for all their internet-scale applications using dynamodb.

As far as this feature is concerned, we would only need a single table called Newsletter, with four item attributes

Id - a string ( partition key ) which will hold the subscriber's email address
Subscribed - a boolean value indicating if the person is subscribed or not.
Validated - a boolean value that will tell if the email is validated. We dont' want to end up reaching out wrong users so this one is important to have in place.
Hash - a random string that will uniquely identify an item in the database, that will be used solely for opting out from receiving notifications.

How to create the table ?

The table will have the following specs:

Email - Partition key ~ String
Subscribed ~ Boolean
Validated ~ Boolean
Hash - Secondary index ~ String - We need this in order to lookup items that are about to be validated later on.

What is api gateway ?

It is a software pattern that sits in front of an application programming interface (API) or group of microservices, to facilitate requests and delivery of data and services.

So the api gateway will be kind of a bridge between the blog and the cloud functions, they will be associated to a particular lambda.

Knowing about this, we can create four(4) endpoints:

POST /newsletter - This one will manage the opt-in for the newsletter.
DELETE /newsletter?h={hash} - Conversely, this one will opt-out readers from the newsletter.
POST /notify - This one will be in charge of notifying the reader via email when a new article is published.
POST /validate - Finally this endpoint is intended to validate recently subscribed readers.

Breaking it down

So the whole idea is:

We're gonna have a frontend newsletter component which will be simply a form that will get the subscriber's email and will hit the POST /newsletter endpoint upon submit.
When the lambda associated to this endpoint is invoked an email will be stored in the dynamodb table along with a hash and a mail will be sent out to this email address to let the user know he must validate his email in order to start receiving notifications. For this take into account that we must have a custom html template in place. There will be a validate link that will take the reader to a validate page in the blog, this page will then hit the GET /validate?h={hash} endpoint of the api gateway.
On the email's template side pertaining to the new article notification there will be a link the reader can click on in case she desires to opt-out of receiving notifications. This link is going to make a DELETE /newsletter?h={hash} request.
Every time a new article is up a new deploy must be done, when it happens a POST /notify request will be made.
So in total we have four(4) lambdas, four(4) api gateway endpoints and two(2) html templates to make, pretty straightforward, right?

It sounds promising, but something that worries me tremendously is how to rate limit the execution of the lambda functions linked to our endpoints, since it could potentially surpass the free tier limits if called deliberately by a malicious person. By fortune, I came across an article where this problem is addressed in a very straightforward manner, by using the following node libraries: async-ratelimiter, ioredis and request-ip

Before including these changes into the lambda function we oughta add a new layer to the lambda function because we're gonna use third party packages now. A lambda layer is a .zip archive that contains libraries and configuration files.

These are the steps we should follow in order to create a new layer and therefore, be able to use external packages in the lambda function:

Create a folder named nodejs.
Within this folder run the following commands to create a package.json file and install the required dependencies:

npm init -y
npm install async-ratelimiter ioredis request-ip crypto-js nodemailer nodemailer-smtp-transport

Compress the nodejs folder to a zipped file.
Now on the aws console, go to the lambda section: layers > create layer. Enter a name for the layer and choose a runtime for node.js and upload the .zip file created.

Since we're gonna be using redis database we can create database in upstash. Once the process is complete you will be given an url of the form rediss://:YOUR_PASSWORD@YOUR_ENDPOINT:YOUR_PORT that you can use to create an environment variable for the lambda function.

It's time to create the email template to inform the reader she must validate her email in order to get further notifications.

<!-- template.html -->
<div style="max-width: 620px; background-color: #fff; border: 2px solid #1D1E22; color: #000; border-radius: 8px; padding: 1rem; display: flex; justify-content: center; align-items: center;">
  <p>
    Welcome to eiberham's blog. In order to receive email notifications you must validate your email address first
  </p>
    <a href="eiberham.com/validate?h={0}" target="_blank" style="display: block; margin: 8px 0; text-decoration: none; border: 1px solid #818890; background: white; padding: 8px; border-radius: 4px; max-width: min-content; color: #818890; font-weight: bold;">Validate</a>
</div>

This is the code pertaining to the aws lambda function in charge of adding items to our dynamodb database, particularly, the one associated to POST /newsletter. Notice how it's limiting the user to perform one single request every five seconds, which is lit. In order to create a hash we're leveraging the crypto-js library. Don't worry about the chunk of code for sending emails for now, I'll talk about it later.

const AWS = require('aws-sdk');
AWS.config.update({ region: "us-east-1" });
const dynamoDb = new AWS.DynamoDB({apiVersion: '2012-08-10'});
const RateLimiter = require("async-ratelimiter");
const Redis = require("ioredis");
const { getClientIp } = require("request-ip");
const cryptoJs = require('crypto-js');
const nodemailer = require("nodemailer");
const smtpTransport = require('nodemailer-smtp-transport');
const fs = require('fs');
const path = require('path');


const rateLimiter = new RateLimiter({
  db: new Redis(process.env.REDIS_SERVER),
  max: 1,
  duration: 5_000,
});

const getHttpResponse = (statusCode, message) => ({
    statusCode,
    body: JSON.stringify({ message })
})

function format( str, ...args ) {
  if ( typeof str !== 'string' ) return ''
  const a = str
  return Object.keys( a ).reduce( ( acc, k ) => acc.replace( `{${k}}`, args[k] ), str )
}

exports.handler = async (event) => {
    const clientIp = getClientIp(event) || "NA";
    const limit = await rateLimiter.get({ id: clientIp });
    if (!limit.remaining) {
        return getHttpResponse(429, 'Sorry, you are rate limited. Wait for 5 seconds')
    }

    const { requestContext, body } = event
    const { email } = JSON.parse(body)

    const tableName = "newsletter"

    try {
        const element = await dynamoDb.getItem({
            "TableName": tableName,
            "Key": {
                "id": { S: email },
            }
        }).promise();
        const exists = element.Item ? true : false
        if (exists) {
            return getHttpResponse(409, 'Email already exists')
        }
        const hash = cryptoJs.MD5(email).toString();
        await dynamoDb.putItem({
            "TableName": tableName,
            "Item" : {
                "id": { S: email },
                "subscribed" : { BOOL: true },
                "validated" : { BOOL: false },
                "hash" : { S: hash },
            }
        }).promise();

        const template = fs.readFileSync(path.resolve(__dirname, 'template.html'), 'utf8')
        const content = format(template, hash)

        const transporter = nodemailer.createTransport(smtpTransport({
          host: 'smtp.gmail.com',
          port: 465,
          secure: true,
          auth: {
            user: process.env.EMAIL,
            pass: process.env.PASS,
          },
        }));

        await transporter.sendMail({
          from: `"Abraham" <${process.env.EMAIL}>`,
          to: email,
          subject: "Welcome - Validate your email",
          html: content,
        });

        return getHttpResponse(200, 'Subscribed successfully')
    } catch(err) {
        return getHttpResponse(200, 'Something bad happened')
    }
};

You can try it out by running this curl command:

user@local ~% 'curl -X POST 'https://<API_GATEWAY_URL>/newsletter' -d '{ "email": "somebody@mail.com" }' -H 'Content-Type: application/json'

Now lets tackle the chunk in charge of validating a user's email. This is simply setting to true the validated attribute of the corresponding item in dynamodb, namely this is the lambda that will be linked to the POST /validate endpoint:

const AWS = require('aws-sdk');
AWS.config.update({ region: "us-east-1" });
const docClient = new AWS.DynamoDB.DocumentClient();
const RateLimiter = require("async-ratelimiter");
const Redis = require("ioredis");
const { getClientIp } = require("request-ip");

const rateLimiter = new RateLimiter({
  db: new Redis(process.env.REDIS_SERVER),
  max: 1,
  duration: 5_000,
});

const getHttpResponse = (statusCode, message) => ({
    statusCode,
    body: JSON.stringify({ message })
})

exports.handler = async (event) => {
    const clientIp = getClientIp(event) || "NA";
    const limit = await rateLimiter.get({ id: clientIp });
    if (!limit.remaining) {
        return getHttpResponse(429, 'Sorry, you are rate limited. Wait for 5 seconds')
    }

    try {
        const body = JSON.parse(event.body)
        const { hash } = body
        const tableName = "newsletter";

        const queryParams = {
            TableName : tableName,
            IndexName : "hash",
            KeyConditionExpression: "#hash = :hash",
            ExpressionAttributeNames:{ "#hash": "hash" },
            ExpressionAttributeValues: { ":hash": hash }
        }

        const result = await docClient.query(queryParams).promise()

        if (!Array.isArray(result.Items) && !result.Items.length > 0) {
            return getHttpResponse(400, 'Not found')
        }

        const element = result.Items.shift()

        const updateParams = {
            TableName: tableName,
            Key: { "id": element.id },
            UpdateExpression: "set #validated = :x",
            ExpressionAttributeNames: { "#validated": "validated" },
            ExpressionAttributeValues: { ":x": true },
            ReturnValues: 'UPDATED_NEW'
        };
        await docClient.update(updateParams).promise();
        return getHttpResponse(200, 'Validated successfully')
    } catch (ex) {
        return getHttpResponse(500, ex);
    }
};

Now let us create the lambda in charge of opting-out readers of the newsletter. This is simply setting to false the subscribed attribute of the corresponding item in dynamodb, namely this is the lambda that will be linked to the DELETE /newsletter?h={hash} endpoint:

const AWS = require('aws-sdk');
AWS.config.update({ region: "us-east-1" });
const dynamoDb = new AWS.DynamoDB({apiVersion: '2012-08-10'});
const RateLimiter = require("async-ratelimiter");
const Redis = require("ioredis");
const { getClientIp } = require("request-ip");

const rateLimiter = new RateLimiter({
  db: new Redis(process.env.REDIS_SERVER),
  max: 1,
  duration: 5_000,
});

const getHttpResponse = (statusCode, message) => ({
    statusCode,
    body: JSON.stringify({ message })
})

exports.handler = async (event) => {
    const clientIp = getClientIp(event) || "NA";
    const limit = await rateLimiter.get({ id: clientIp });
    if (!limit.remaining) {
        return getHttpResponse(429, 'Sorry, you are rate limited.')
    }

    const { queryStringParameters } = event
    const tableName = "newsletter"
    const queryParam = queryStringParameters || null
    if (queryParam) {
        const hash = queryStringParameters.h || null
        if (!hash) return getHttpResponse(500, 'Incorrect query param')
        const element = await dynamoDb.getItem({
            "TableName": tableName,
            "Key": {
                "hash": { S: hash },
            }
        }).promise();
        if (element.Item) {
            await dynamoDb.updateItem({
                "TableName": tableName,
                "Item" : {
                    "id": { S: element.Item.email },
                    "subscribed" : { BOOL: false }
                }
            }).promise();
            return getHttpResponse(200, 'User unsubscribed successfully')
        } else {
            return getHttpResponse(200, 'User does not exists')
        }
    }

Now that functionality to subscribe and unsubscribe readers is in place, we're all set to take care of the last part backend-related, sending mails to subscribers every time a new article comes up.

How to send emails ?

Easy peasy, by using nodemailer.

Nodemailer is a module for Node.js applications to allow easy as cake email sending. The project got started back in 2010 when there was no sane option to send email messages, today it is the solution most Node.js users turn to by default.

So this time around we're going to create a separate lambda function which will be responsible of notifying every user with an active subscription as soon as a new article is published. But before that we need to have something else in place though, a mail template!

<!-- template.html -->
<div style="max-width: 620px; background-color: #F0F0F0; border: 2px solid #1D1E22; color: #000; border-radius: 8px; padding: 1rem; display: flex; justify-content: center; align-items: center;">
  <p>
    <strong>Dear Subscriber</strong>, <br/>
We’ve just published a new article titled {0} <br/>
We believe you might find it insteresting to read. You can read it on our website by clicking the button below
    <a href="eiberham.com/{1}" target="_blank" style="display: block; margin: 8px 0; text-decoration: none; border: 2px solid #41a85f; background: white; padding: 8px; border-radius: 4px; max-width: min-content; color: #41a85f; font-weight: bold;">READ</a>
    <a href="eiberham.com/optout?h={2}"><span style="font: bold 11px Arial;">Click here if you no longer want to receive these notifications</span></a>
  </p>
</div>

I know it's a shitty template but don't sweat it you can tweak it to accomodate your likings, anyway, with the email template in place we can follow up with the lambda function for delivering emails. Quick heads up, if you use gmail you must be aware they shot down less secure apps so using gmail with nodemailer now will throw an error, unless you do two things:

Enable two steps verification on your gmail account.
Create an application password on your account settings.

Since sensitive data like email credentials are required in order to send emails we must create environment variables to be used by our lambda function, just a heads up, simply head over your lambda configuration and create them.

Following up, our lambda function will be open and pretty much akin to the previous one and it's going to accept only post methods.

const AWS = require('aws-sdk');
AWS.config.update({ region: "us-east-1" });
const docClient = new AWS.DynamoDB.DocumentClient();
const nodemailer = require("nodemailer");
const smtpTransport = require('nodemailer-smtp-transport');
const fs = require('fs')
const path = require('path')

function format( str, ...args ) {
  if ( typeof str !== 'string' ) return ''
  const a = str
  return Object.keys( a ).reduce( ( acc, k ) => acc.replace( `{${k}}`, args[k] ), str )
}

exports.handler = async (event) => {
  const tableName = "newsletter"
  const { body } = event
  const { title, slug } = JSON.parse(body)
  try {
        const params = {
          TableName: tableName,
          FilterExpression: "#subscribed = :subscribed",
          ExpressionAttributeNames: {
              "#subscribed": "subscribed",
          },
          ExpressionAttributeValues: { ":subscribed": true }
        };

        const recipients = []
        let items;

        do {
            items = await docClient.scan(params).promise();
            items.Items.forEach((item) => recipients.push(item.id));
            params.ExclusiveStartKey = items.LastEvaluatedKey;
        } while (typeof items.LastEvaluatedKey != "undefined");

        const template = fs.readFileSync(path.resolve(__dirname, 'template.html'), 'utf8')
        const content = format(template, title, slug)

        const transporter = nodemailer.createTransport(smtpTransport({
          host: 'smtp.gmail.com',
          port: 465,
          secure: true,
          auth: {
            user: process.env.EMAIL,
            pass: process.env.PASS,
          },
        }));

        const info = await transporter.sendMail({
          from: `"Abraham" <${process.env.EMAIL}>`,
          to: recipients,

          subject: "A new article has been published",
          html: content,
        });

        const response = {
          statusCode: 200,
          body: JSON.stringify({ message: "email sent successfully" })
        }

        return response;

  } catch (err) {
    return {
          statusCode: 200,
          body: JSON.stringify({ message: err })
    }
  }
}

In order to test it you can perform the following curl command:

user@local ~% 'curl -X POST 'https://<API_GATEWAY_URL>/notify' -d '{ "title": "title", "slug": "slug" }' -H 'Content-Type: application/json'

At this point what is left is the frontend part, which is simply a controlled form component that triggers the lambda function when submitted. We could make this component a bit smaller by moving the submit function to an util file or a custom hook for example, but for simplicity we will leave it as is.

Notice the env variable. This is holding the api gateway endpoint url in charge of signing up reader to the newsletter.

// Newsletter.jsx

import React, { useState } from 'react'

import './styles.scss'

const Newsletter = () => {
  const [ email, setEmail ] = useState('')
  const endpoint = process.env.GATSBY_SUBSCRIPTION_ENDPOINT
  const onEmailChange = (e) => setEmail(e.target.value)
  const isValid = email => /^\w+([\.-]?\w+)*@\w+([\.-]?\w+)*(\.\w{2,3})+$/.test(email)
  const onSubmit = async (e) => {
    e.preventDefault()
    try {
      const data = { email }
      await fetch(endpoint, {
        method: 'post',
        body: JSON.stringify(data)
      })
      setEmail('')
    } catch (err) {
      console.error(err)
    }
  }
  return (
    <section className="newsletter">
      <form className="form" onSubmit={onSubmit}>
        <div className="container">
          <p className="container-text">
            Do not miss out. <br />
            If you want to get the latest content via email subscribe to the newsletter
          </p>
          <div className="fieldset">
            <div>
              <input type="text" name="email" value={email} onChange={onEmailChange} required />
              <button type="submit" title="subscribe" disabled={!isValid(email)}>
                <span>Subscribe</span>
              </button>
            </div>
            {email && !isValid(email) && (
              <div className="error">The email format is incorrect</div>
            )}
          </div>
        </div>
      </form>
    </section>
  )
}

export default Newsletter

The styling is pretty straightforward, it's just modifying the text input and the submit button a little bit to make them wider and fancier, you can style it all differently to accomodate your needs.

/* styles.css */

.newsletter {
  align-items: center;
  border: 1px solid rgba(49, 46, 129, 0.75);
  border-radius: 8px;
  display: flex;
  justify-content: center;
  margin-bottom: 2rem;
  padding: 1rem;
  width: 100%;

  form {
    .container {
      align-items: center;
      display: flex;
      flex-direction: column;
      justify-content: center;

      .container-text {
        align-items: center;
        display: flex;
        justify-content: center;
        width: 100%;
      }

      .fieldset {
        display: flex;
        flex-direction: column;
        justify-content: flex-start;
        padding: 8px 0;
        width: 100%;

        input {
          background: #fff;
          border: 1px solid rgba(34,36,38,.15);
          border: {
            bottom-right-radius: 0;
            radius: 0.28571429rem;
            right-color: transparent;
            top-right-radius: 0;
          }
          box-shadow: none;
          color: rgba(0,0,0,.87);
          flex: 1 0 auto;
          font-family: Lato,'Helvetica Neue',Arial,Helvetica,sans-serif;
          line-height: 1.21428571em;
          margin: 0;
          max-width: 100%;
          outline: 0;
          padding: 0.6em 0.7em;
          text-align: left;
        }

        button {
          background-color: rgba(49, 46, 129, 0.75);
          border: {
            color: transparent;
            radius: 0 0.28571429rem 0.28571429rem 0;
          }
          color: #fff;
          line-height: 1.8rem;
        }

        .error {
          color: red;
          font-size: 0.875rem;
          padding: 8px 0;
        }
      }
    }
  }
}

How do we trigger the reader notification ?

Since the blog is hosted on github we can easily create a github action that sends a post request to the /notify endpoint as soon as the code reaches the production stage. So let's go ahead and craft the continuous delivery yaml that will make it happen.

So the lambda function is supposed to receive two parameters: the title and the slug, right ?

Well, by stablishing a commit convention for new articles we could somehow send both parameters in the commit message. It seems odd but based on my knowledge or lack thereof, that what comes to mind for the time being. I'll come up with something better down the road, let's keep with this solution for now.

So commits with this format {"title":"title","slug":"slug"} will hit the /notify resource. Therefore it's important to either create a script to automate this task or remember to commit in this format every time a new article is up.

# cd.yml

name: blog continuous integration workflow
on:
  push:
    branches:
      - master
jobs:
  deployment:
   runs-on: ubuntu-latest
   steps:
    - name: notify new article
      id: notify
      if: startsWith( github.event.commits[0].message, '{' )
      env:
        msg: ${{ github.event.commits[0].message }}
      uses: fjogeleit/http-request-action@v1
      with:
       url: ${{ secrets.NOTIFY_ENDPOINT }}
       method: 'POST'
       data: ${{ $msg }}

What we're doing is telling github every time there's a push on the master branch a custom job is executed. If the commit message starts with a curly brace it is going to perform a post request to the POST /notify endpoint with the commit message as payload

Bash script

Now, let's create the bash script that will automate the commit submission for new articles. The script will receive two arguments, first the title, and lastly the slug.

#!/bin/sh
# eiberham-blog-auto-commit-script

TITLE = $1
SLUG = $2

git commit -am "{ title: ${TITLE}, slug: ${SLUG} }"
git push

exit

Really simple, now every time a new article is wrapped up we can execute the script.

Conclusion

Well dear reader, you have witnessed how you can take this approach to create a newsletter subscription service on your site using amazon web services and react.js, certainly it's not the best approach as there are a lot of options for doing this all over the internet. You can take this as a starting point to get your head around some gotchas involved in the topic and come up with something that suit your needs later on.