SSR but just for bots

2022-05-24

One of the projects I’m working on recently needs to have proper SEO, namely, the following things need to be accomplished:

  • Certain pages should show up in Google Search results like this:

    CleanShot 2022-05-24 at 16.51.47@2x

  • Certain pages should have proper preview cards on social platforms like this:

    CleanShot 2022-05-24 at 16.49.47@2x

The most obvious solution is probably converting this project to a server-side rendering app, since it’s written in Vue.js, there’s Nuxt.js the Vue SSR framework for us to use.

But this project has not been designed with server-side rendering in mind, it would probably take at least a couple of months to convert it to a full server-rendered Nuxt.js app, and deal with new problems SSR brings us in the future.

It’s just not worth it for those two requirements.

But we do need SSR, and in fact, only SSR can solve my problem, so I went back to the problems I have and realized that I just need SSR for rendering <meta> tags so Google and crawlers can index the page without running the JavaScript.

Now the solution is pretty simple:

  1. request comes in
  2. use some server-side code to detect bots
  3. if bot: fetch our API to get necessary data to render <meta> tags, and insert them into the HTML template (generated by Vite), it’s fine that it’s a little slower for bots
  4. if not: just serve the HTML template, no API calls, faster for real users

Here comes another problem in steps 2 and 3, we need to run server-side code in order to detect bots and fetch data, which means the website is no longer a static site that is distributed and cached through a global CDN. Technically I can just create a Node.js server (or serverless function) and deploy it close to the target users (in 🇯🇵 Japan), but it’s not possible with Firebase Hosting, which only allows serverless functions in us-central-1, so I decided to move it to Vercel.

Instead of deploying it as a Vercel serverless function in Tokyo, Japan, I went another route, the Edge Function, which is basically some server-side code running close to your users globally, so you get the benefits of static (speed) with the power of dynamic (customization).

Vercel allows you to add an edge function as middleware in front of your website, the edge function looks like this:

pages/_middleware.js
export default function middleware(req, ev) {
console.log("Edit and run at the edge!")
return new Response({
ip: req.ip,
geo: req.geo, // this will spin the globe!
ua: req.ua,
})
}

If you don’t want to intercept the request you can return a response with the header x-middleware-next: 1 to pass it through to your app, in Next.js they provided a helper function NextResponse.next() for this.

Technically both the Serverless Function deployed in Japan and Edge Function (worldwide) will work for this project, but anyway, I’ve been wanting to experiment with Vercel Edge Function in a Vite project for a while, so let’s freaking go XD!

I went out of my way to create a Vite plugin for this: https://github.com/egoist/vite-vercel, Edge Function right in your Vite projects! I have another similar project vite-plugin-mix which is for Vercel Serverless Function instead.

While Edge Function is handy, it also comes with limitations (duh!):

The function needs to return a response in less than 1.5 seconds, otherwise, the request will time out. — Vercel Docs

Sometimes the API call I use takes longer than that, but since Edge Function is built upon standard Web API, I can work around this by using a TransformStream:

import { MiddlewareResponse } from "vite-vercel/server"
export async function middleware(req, event) {
const isBot = checkIsBot(req.headers.get("user-agent"))
if (isBot) {
event.waitUntil(
(async () => {
const html = VITE_INJECTED_HTML
const { readable, writable } = new TransformStream()
const metaTags = await getMetaTagsFromOurApi()
const writer = writable.getWriter()
const encoder = new TextEncoder()
const result = html.replace("</head>", `${metaTags}</head>`)
writer.write(encoder.encode(result))
writer.close()
})()
)
return new Response(readable, {
headers: {
"content-type": "text/html; charset=utf-8",
},
})
}
// Same as NextResponse.next() in Next.js
return MiddlewareResponse.next()
}

This allows me to return the response as soon as possible, and continue the work after that, the maximum duration for an Edge Function execution is 30 seconds so this should be fine.

You may notice the VITE_INJECTED_HTML variable in the above code, you can get it like this in vite.config.ts:

export default defineConfig(({ mode }) => {
// VITE_VERCEL_BUILD is an environment variable set by vite-vercel
const VITE_INJECTED_HTML =
mode === 'production' && process.env.VITE_VERCEL_BUILD
? fs.readFileSync('dist/index.html', 'utf8')
: ''
return {
define: {
VITE_INJECTED_HTML: JSON.stringify(VITE_INJECTED_HTML),
},
})

And that’s it, problems solved in a few days instead of months, now my only concern is the cost, if it costs significantly more than the serverless function alternative, I might switch it to the other solution.

Anyway, thanks for coming to my TED talk.