Luis Ball 0:16 Hi, my name is Luis ball. I’m a senior software engineer at images. Today we’re going to talk about why you should build your own imaging pipeline, and we’re going to look at some practical ways to do that.

By the time we’re done, we’re going to have something like this. We’re going to have a deployed next JS starter, where our images are being served from a image optimized API that we control, and that API will fetch a remote image, resize or crop, reformat the image, and send it off to the browser. And in this case, what we want to end up with is also some caching so we don’t generate the images and optimize the images every single time they’re requested, but we do it just once and anytime we change a parameter or a file format. So let’s dive in on how we can do this, different optimization techniques, different libraries we can use and end up with a deployed next JS starter like this. Let’s dive in now. You might still be thinking, well, this doesn’t apply to me. Well, bear in mind that 70% of all mobile websites, or all websites rendered on a mobile device have their largest contentful paint on the page as an image. So the thing that most impacts that initial performance is an image for 70% of sites on mobile. This is actually up to 80% on desktop. So believe it or not, this is probably going to benefit you. Optimizing images is something that generally, all of us need to be doing as much as we can, as often as we can.

But how do you actually optimize an image? There’s a lot of ways to do this, but there’s four things you’re typically looking for. Let’s first start by talking about compression and file formats, because they go ahead and all

right now let’s take a peek at how we can impact the file size with compression and file formats. So when I move the Quality slider down here, what we’re gonna see is our file size will change. So I’ll put it down to about 50, and we’ll see what happens. We’re at 778, kilobytes. So we’ve lowered the quality. There’s more compression going on, and the resulting image still looks pretty darn good. However, we’d like to get more file savings. And in order to do that, I could put the quality down all the way to maybe, I don’t know, 10 or 12 or 15, but what you start to notice is just more artifacting. You’re losing colors and detail in the image, which isn’t great, right? We want this to look good, which is where the modern file formats come into play. Because with these, even at 75% quality, we’re getting incredible compression here, 655 kilobytes. But I could also be really aggressive with this quality slider, and you won’t see the kind of artifact that we saw before. We’re at 182 kilobytes, temp quality, and the image still looks incredibly good. Now let’s take a peek at what happens if we combine this with resizing and cropping. So we’ll go back to browser J bank, we’ll put it back at 100 and let’s see what happens if we just resize the image. So right now, it’s 4000 by almost 3000 if I bump this down to, let’s say, 1000 by almost 1000 you know, 1200 by 800 we’ve made it much smaller, right than in the five ish megabytes that it was before. Now it’s just one and we haven’t lost too much detail, but you can see this, this quite a bit of detail we’ve lost. Now that would be okay if we render the image in the smaller container, because the difference here won’t be really be all that noticeable. And where this really begins to matter is when you combine it with compression, right? Because if I go in here and now I say WebP, and I put this down to 20, all of a sudden, the image still looks good at this container size, but look at the file size. It’s 30 kilobytes. So when we combine the approaches of resizing with a modern, modern file format with compression, you can get some pretty incredible file size savings. And so this is why using all four of these together is super useful. Now, the one thing we haven’t looked at here is cropping, because if I only needed the immediate area around the surfer and none of the wave, I’d give it even more bytes shaved off of this. But for now, let’s just keep this in mind, right? Combining all of these together will give us significant file size savings, all right? Well, that was a lot of the why. Now let’s dive into the how, using IPX.

IPX is a Node js image optimizer. It uses svgo and sharp under the hood, it’s used by Knox image and Netlify. It’s very easy to use. It’s open source. You can self host.

It, and all of the optimizations that it enables are controllable via the URL.

Some examples of how they use this with nuxt image, if you don’t have an image CDN, the default transformations that are done with that component are with IPX and with Netlify. If you deploy next year Netlify, you actually have the option to do your image transformations using IPX behind the scenes.

Let’s take a look at how it works. So IPX enables you to do some common operations using the image URL. In this example, we have our site, my site.com and it has some images, presumably under the file path of slash images. Slash whatever image I’m requesting. We’ve configured it so that when you visit the images route, instead of serving the file directly, you’re going to hit the IPX server instead, and it’s going to apply the transformations or optimizations that we want it to. So what kind of optimizations can we do? Well, we can reformat the image, we can resize the image, we can also crop to cover a specific dimension for wherever it is the image is being rendered. And this is just a subset of the things that you can do with IPX. So again, IPX is a no JS image optimizer. It runs on a server. You can configure it to intercept routes so that when an image is requested, instead of serving that image, you serve a transformed and ideally optimized version of the image that’s being requested. Now let’s take a look at how IPX works under the hood, a little bit specifically, starting by talking about what sharp is.

So what is sharp? Well, sharp is another node js image processor, except it’s actually what powers IPX. Use sharp to reformat and resize and do other things with images. But the really cool thing is that it interfaces with a native code in this case, C Plus Plus, and it binds to a library called libbips. So here’s an example of how using sharp actually looks you give it an input, you give it some operations, you then output the now resulting image, and then you can do some other things. Once you have a result in the transformed image,

sharp, again, is what powers IPX behind the scenes. And the power of sharp comes from the fact that it binds JavaScript code with the native code. So let’s talk about that native code. Let’s talk about what lip VIPs is.

So what the heck is lip VIPs? Libvips is another image processing library, but this is like the grandfather. This is the head honcho behind the scenes of both IPX and sharp, and it surprise, surprise is used to reformat, resize and do all another other kinds of transformations to images. One of the things that makes this very popular is that it’s also got a lot of language bindings, everything from C Plus Plus to PHP.

So again, just to recap, lib dips is what power sharp, and sharp is in large part what powers IPX. So let’s now step away from the theory and actually make an IPX server that can serve our optimized images. We’re going to be doing it first just as a standalone server, and then we’ll look at how to use it inside of some front end frameworks like next js.

We’re going to want to install the IPX Hano and button packages. We’re going to be using bun here, but you can use whatever package manager you like, NPM, PMPM, whatever it is, just make sure that these are the packages that we’re installing, because we’re going to need them. So first off, create IPX with create IPX, we actually configure our IPX instance. Here. We’re going to pass in some options with the ones we care about for right now are storage and HTTP storage. So with storage, we can use the IPX Fs storage to allow IPX to access the file system and serve images from any number of directories where we have images. So on this server, we’ll have images on the static directory that might be a little bit different for you, the HTTP storage configuration that one allows us to use the IPX HTTP storage function to allow IPX to optimize remote images. And so here we configure the domain of where.

Where those remote images live, and which limits what remote images can get optimized by our server is somebody tries to request an image that doesn’t have this domain, we’re going to see an error and it won’t optimize it. We’ll dive into all of this in just a second, but bear in mind, this is how we’re configuring IPX to tell it where our images are coming from, either from the static folder or from a remote storage that lives under this domain. Now that we’ve configured IPX, we can dive into configuring hanno to actually serve our images anytime they’re fetched. So to configure hanno

First things first, we just create a new hanno instance. I’m going to call it image, but you can call it whatever you like. And then I want to register the optimized route on that Hano instance, and I want to handle requests that come into it. So when a request comes into the optimized route, I want to grab that URL that was requested, and then I want to replace and remove the optimized part of it so I can give that to the ipx web server

using a request object. So why do I need to wrap the URL in a request object? Well, this is what the IPX web server expects us to do. It uses properties from the request object to make all kinds of decisions. Just don’t forget to wrap that URL in a request object before you give it to the ipx web server. Why bother register the optimized route? Well, I might have other routes in my server. You can name this whatever you like. I’ve seen a lot of people come in here and do underscore IPX. It’s a pretty common convention, but just for the purposes of this demo, we’re going to go with optimize it works for me. After we’ve created our optimize route, we’re going to actually mount that route to our main Hano app. And then once we’ve done that, we can use buns serve utility to actually create the web server tell it to use our Hano apps fetch

request handler for any incoming fetch call, and we’re going to expose the server on port 3000 so again, just a quick recap. We instantiated IPX, we went ahead and defined our Hano routes. We created a Hano app and mounted those routes to it, and then we told buns serve utility to use the Hano app for any fetch calls and export, excuse me, expose the whole thing on port 3000 so once we’ve done all this, the last thing that is happening here, we’re just logging to make sure that we knew, we know, we, you know, reached the end of the file and that our server is running. So let’s go ahead and run the server now, and we can see our server is running. We’ve got the helpful, helpful message there. Let’s go ahead and request an image so I know this image exists. You can see it here. There’s the file path right rev on least 24 and I have this underscore in front. Now, this is a convention in IPX. This is basically telling the IPX transformation not to do anything. This is a kind of like a no op, if you will. So what’s going to happen? If you remember, when we actually request this image, we’re going to replace the Optimize and remove this entirely. Essentially, this is what IPX will see, and it’s going to look at the first part of the path and say, Okay, I’ve got a no op transformation here. Let me just actually serve up the underlying file path. And I know where that lives, because I’ve been told that that’s where the storage of the directory is under static and again, like this is where that image is. So anyway, let’s go ahead and request it. There is our image. Now, this is not very impressive. We haven’t really done anything to it. We’re just serving an image. But we are, you know, serving an image. Let’s do some cool stuff, right? Let’s go, go ahead and actually do some of the things that IPX is really good at doing, which is resizing, for example. So let’s give it a w2 100 transformation, and now our image gets resized. And we can make it smaller, like we just did there. We can make it a little bit bigger. There is some upscaling we can do. We could use the size parameter here. So if I say s and then we give it, let’s say 200 by 200

then it will actually crop this image. And then you can also use this in combination with the fit parameter to say something like fit fill or fit cover, and this changes the way the crop of this image is actually taking place.

Oops, I did not mean crop. I meant cover. There we go. So yeah, now you see all our transformations are taking place. All we had to do was put them in the part of the URL path that comes directly before the actual file name that we’re trying to serve up. And this is all pretty cool. However, there is something.

We set up, which is also fantastic, which is the ability to optimize remote images. And this is much more common, especially as you start to generate lots and lots of images on your pages. You’re typically not committing them to source control. You’ll be storing them in s3, GCS, whatever it might be. So let’s show you an example of that. Let’s imagine that I want to request an image that lives here. Let’s say colon, slash, slash, execute test

Amsterdam, I think is the image.

And there you are. We’ve just optimized well in this case, really just served a remote image, but we can optimize it. That’s what makes IPX so cool, right? We can go in here, say, w2 100, and crop this image, which we isn’t even on our server. It’s hosted somewhere else, but we’ve gone, requested it and optimized it and then returned it in the crop or in the resize that we wanted. So just to take a quick second here, the way this is working again, is when we create our IPX instance, we tell the HTTP storage configuration to use IPX HTTP storage, and then we give it the domain and say, Yeah, this is the domain you’re allowed to fetch from. Because if I came in here, let’s go to Unsplash, right? And if we go to Unsplash and we say, copy any of these image URLs, if I try to serve this image, watch what happens.

I get a forbidden host because I’ve told that you’re not allowed to optimize images from any other domain than the one that I’ve configured you to work on. So this is pretty good. This is going to work very well for some limited use cases. And we’re going to dive into the next steps here. Next steps are hooking this up to our front end, maybe a next js application, and actually being able to optimize our images at request time on that application from our IPX server. There’s some catches. You can run into some some trade offs to consider, but we’ll dive into all of that next.

All right, let’s take a look at how we can add IPX to a next JS project. Now this is going to be a little bit different than what we just did. We’re not going to have a standalone server that runs next to our next js application. Instead, we’re going to leverage the app directory’s API routes to on request transform images, yeah, using an API API route, a lot of this will carry over, but just note that this is slightly different. So we’re going to start here. Let’s create a route.ts file that will use IPX to optimize an image anytime this route is hit. So first things first, we’re going to bring in IPX just like we did last time, only one small difference. We’re going to use an alias. And what this allows us to do is, instead of having to type in the entire domain every time or URL every time we want to request a remote image, we can just type in slash image slash whatever path we want, because that’s the alias we’ve configured for that domain. It’s a little bit of a quality quality of life change. Everything else is the same. We have a storage and an HTTP storage. In this case, we’re going to be pointing to the public directory, because that’s where next JS puts these assets. Our domains configuration has not changed.

Another call out about route files in next js is App Directory is we have to use the Node js runtime here because we want access to the file system, and I believe also because for IPX to work, we just have to run in no JS runtime. I don’t think we can use this in the edge runtime. This will be a serverless function, so keep that in mind. Anyway, let’s go into the get function. So when a get request is made here, the first thing that we’re going to do is pull the image path from the request URL. I’m doing it pretty naively here. I’m just assuming that it’s going to be within a specific part of the path. This might look a little bit different for you. If your API route is different than mine,

then we’re going to pull the search parameters that we care about. So there might be a lot more. You might want to do something more complicated here, but for now, I just care about with high format and quality. So I’m going to pull them from the request and store them so that I can apply them as IPX transformations. And to do that, I create an operations object and where, essentially it’s just key value pairs. The key of width points to the value of width that was set in the search params. And once I have this operations object, we can then go and call IPX with the path of the image that we want to render. This can be again, a.

Local or Remote, and then the operations that we want to perform on this image, then we just call process, and we have a processed image. Now we need to return the data that we get back from IPX in our next response, and that’s what we do here. And one other call out is because we are not using the built in next js, image optimization. We want to be careful to establish a caching strategy, and we’ll dive into this in a second. But it’s important, because if we don’t have any kind of caching, every single time an image is requested, it is going to be generated on the fly. And this can get very slow and it can get very expensive. So the part of the way we do this is we use the content type header. This is part of our caching strategy. It essentially tells us whether or not we’ve changed the file format that we want to render, in which case we want to run to render the old image we want to render a new one. And we’re also going to use a cache control header. And here we’re going to set the max age to a minute and a stale while revalidate to a minute as well. This is very short. You can make this much longer, but we’re just doing it this short, so it’s easy to see when the cache gets invalidated and that it’s working the way that we want it to. You probably want to bump this up to something that makes more sense to you, some very basic error handling here. You may want to do something a little bit more robust if you are going to ship this anyway. This is our route.ts, file. This is enough to start making requests to this API, but it’s not really integrated into next js. We need to make some other changes to be able to use the next image component with what we’ve done here. So let’s take a look at that.

We’re going to go into our next config file. So in here we need to go into next config and set the images loader to custom, give it a loader file, which we’ll make in a second, and then also set up the remote patterns configuration. This is helpful to prevent abuse. Essentially, it won’t try to optimize images that don’t match this protocol and host name. We’ve already set up some of this with IPX, but it’s it’s always good to put this in here.

Also just validate so on build time. I believe if you don’t use the right host name, it gets angry. That’s that can be really useful anyway. Now that we’ve configured the next config, the next thing is to actually make this loader file, and this is what nextjs will use whenever you’re using the next image component. So let’s take a look at the loader file. It’s not very complicated. All that we’re really doing here is saying, Well, do we have a remote image? If we do, then we want to encode that remote image. In our case, we’re going to be using the alias anyway, so that won’t be necessary. If we don’t have a remote image, then check if it’s a relative file path. If it’s not a relative file path, then, you know, go ahead and just return the source attribute. Otherwise, I think what all we’re doing here is just deleting the prepending slash.

Now that we’ve done those things, the image path, we can make our URL. And our URL is going to be just our API route. It’s going to be API optimized, and then the actual image path that we want to request and then transform. And also, we have to go ahead and get those query params that we care about and set them so that the URL that we request also has them. So we create a search params object, and then we return A concatenated string with the actual path of the image that we want to request. So in this case, slash API, slash optimize image path with the query params that we want to set for it. We’re just using width and height, excuse me, width and quality. But you could be setting a lot more here, if you wanted to. This is just enough for now to get us started. So now that we’ve set up the image loader, we’ve set up the next config, and we’ve set up the wrap file, let’s just take a look at the page file to see what it looks like. This is the Create next app boiler plate that comes when you run that in the CLI. And all I’ve done is grabbed the main image here and replaced it with our Amsterdam dot jpeg image that we saw before. I’m using our alias. Let’s go ahead and update the alt text to say Amsterdam. Uh, I don’t know, at night or something, we’ll save that. And now let’s go ahead and run our server.

Right. We’re running a local host, 3000 let’s refresh. And here we are. We see our image as well as some other static assets that we’re serving. Let’s go ahead and inspect it and see what that looks like. And there it is. We’ve got our slash API, slash optimize slash image, X, Amsterdam, dot, jpg image. And if we take a closer look here at the server logs, I’m logging the data that we get back. So you can see there’s a few SVGs that came through. I think these are the icons and whatnot. I’m interested to see there we are. We get a buffer for the Amsterdam dot JPEG, and that’s what you see when you go to the route file, and that’s what you see in this console log. Just wanted to point that out, not this one, sorry, this one case, you’re interested in how that’s all coming through, how we get that data. You can actually see some of it come through here.

And obviously we’re in development mode. So if we go to the Network tab and we just refresh like all of this cache is disabled, all of this will be generated every single time. So if we go here and we just go by image, let me keep refreshing. You can kind of get a sense for how long it takes to do this generation each time something gets requested, you know, and it’s pretty fast. 155 milliseconds. Obviously, there’s no latency because we’re literally on the same machine. But, yeah, that’s it. I mean, there’s not much more to do here. We’ve set up our route TS files so that we can handle API requests for a remote image or a local image. We set up our next config so that we could use a custom loader, and then we also set up our custom loader so that we could use the next image component in our page file, which we can see here, and that’s all there is to it. Now I do want to dive into deploying this to production and then seeing what kind of intricacies we run into, some cool things that we can actually observe about how the cache is working and some other things. So let’s take a look at that. So on the left hand side, I’ve got my next JS logs from vercel, and on the right hand side, I have the actual next JS deployment. Let’s go ahead and hard refresh the page. And now we’ve just requested all the images again, and I want to hit this filter that says cache hit. And you can see that all the images are actually hitting a cache. And let’s take a look at this one. For example. This is this Amsterdam image, you can see the path, the user agent, all that good stuff. We also see the search params that it was requested with the fact that it ran on no JS runtime, etc. And if we take a closer look here at the actual network tab, I’ve disabled the cache so everything will be fetched again. And if I do a hard refresh, we can take a look at the response times.

Some of these smaller images, they’re within 60 milliseconds. But if we could look at some of the larger ones, this Amsterdam one, for example, this took about 100 milliseconds. And that’s, you know, pretty good, but it can be better. Here’s 97 ideally you want that to be anywhere in the vicinity of 60 milliseconds. But I mean that all depends on where you are relative to where your deployment is and whatnot. But that’s honestly pretty darn good. And again, this is all cached, so if we go to the live view here, you’ll see that I’m just constantly hitting the cache of the image that was generated when it was first requested. Now if I were to, let’s close the inspector here. If I were to modify this and change the parameters a little bit. So let’s, let’s assume I think it’s gonna switch sources. Did it pick? Did it pick 600 by 400 so here’s 640 that’s the one. Let’s change the Q 75 and all that stuff. Let’s make this like,

I don’t know, 100 and see if we can’t get this to request a new image. There it is. You see how much longer that request took. It took about 300 milliseconds. That’s because it’s actually been generated for the very first time, because it wasn’t cached. And here you can actually see. Be the miss of the image we requested right here, because it had to get generated on the fly. Now I have a good connection, so that still felt really fast, but that’s one of the things that you know you’ll have to sacrifice when you’re considering generating these variations at request time is the very first time. It’s always going to be a little bit slower, and from that point on, it’ll get cached. Now, another thing you have to bear in mind is every single cache hit on vercel will be charged. It’s It’s not like you know, because it’s got cached. If it gets requested 1000 times, you just don’t get billed. So one of the things you’re going to want to do if you deploy this somewhere is probably put a CDN in front of all of the image paths for this site. So and configured the same way that we configured the cache for the vercel deployment so that they expire at the same time. But at least that way, the vast majority of the cache hits will happen at like a Cloudflare CDN, where there’ll be a lot less expensive because you just won’t be hitting a price tier that charges you for every time that you actually go and hit a cache. Now that being said, this was really quick and painless. It got us to a point where now we have image optimization deeply tied into the framework that we’re working in. We can control the entire optimization pipeline and maybe even make some extensions to it, and we can put things like a CDN in front of it to control our costs and make sure that we’re not always regenerating the images.

One of the things that we have to consider is, even though we’ve done all of that, there’s still the potential for abuse, like you just saw. If I go in here and inspect this image and change parameters, I could be generating variations on the fly endlessly, and you’ll have to mitigate this, and there’s several ways to do that. There’s also the potential that our little node function, for some reason, fails. Maybe we just happen to have an outage of your cell, or maybe we’re trying to run too many first time optimization requests at once. And that can get very expensive, right? Even if it is just that first request, if I have, you know, a lot of user generated content, a lot of UGC, and you know, 1000s of images are getting generated and requests it all at once. You could see this driving up my costs, because these are serverless functions.

So again, a lot of caveats here, but this is a pretty good starting point. It’s very inexpensive if you don’t have that many images, if you put a CDN in front and if you take some common sense steps to just make sure that people aren’t spamming your optimization endpoints.

I Let’s talk about some of the limitations of our self hosting approach. That we covered. First and foremost is the caching conversation that we started. Although vercel does offer caching, as we talked about, the pricing may not work for you, so you may want to put a Cloudflare cache or something like that in front and I want to show you a little bit of how you could do that. So with CloudFlare, you have this page rules, rule, and with this, like this great article by I don’t know how to say this st forengine shows us is you can use a page rule to tell Cloudflare Hey for this particular route, I want you to just cache everything and cache it for this amount of time, and you can configure this to be to match the cache that you’ve configured on vercel. So if it for whatever reason, it misses the Cloudflare cache, then it should hit the vercel cache. And obviously, if it doesn’t happen there, it would actually transform the image at request time. But again, I just wanted to call this out, because this will help control costs, and it is a source of complexity, because you need to control the invalidation of your image. Let’s say you change a file format, maybe you re upload it to the same file path. You’ll need to find ways to evaluate this cache. It might even be manual, or maybe you have some kind of CLI or automated process to do that for you, but again, caching will help you control costs and will help you control how many times an image does have to be reprocessed, re optimized when it gets requested.

Something else I want to call out is security. So when you’re processing images on your servers, you’re reformatting them, or you’re even processing, maybe UGC user uploaded an image to one of your buckets, potentially, or you’re fetching it from their buckets. You’re also introducing an attack vector, and an example of how this could impact you is what happened with the image magic exploit a while back, and essentially, there was a way to escape.

Uh, the execution context so that you could run arbitrary code on anybody’s servers. And this affected quite a few folks. Now this, I believe, has since been patched. This particular exploit. I mean, it’s been a while, but there’s so many others, especially for older file formats, which can be difficult to patch because of backwards compatibility reasons. So just be in the know about these types of exploits that can and do happen, and you need to stay on top of them, especially if you have a lot of UGC content that you don’t control, being uploaded to your servers and then processed by your image processing pipeline. So those are two really common caveats worth keeping in mind when you’re going to be self hosting these pipelines, that was a lot. It was really fun to go through it. Hope you enjoyed it. Please let me know. If you’ve got feedback, you can find me on Twitter. I’ll leave my handle here. You could also, ideally, when this makes it to YouTube, leave a comment and let us know what you think. Thanks so much for watching. And a small call out here is self hosting is great. Gives you a lot of control, but you could do almost away with almost all this complexity. If you do go with an image CDN that takes care of all the hairy details like caching, invalidation of the cache and security for you, just make sure you always price out what option makes the most sense for what you’re working on. Thanks again for coming and watching and letting me know what you thought you.