The Web App and the Web Site Should Be Friends

The web app and the web site should be friends,
Oh, the web app and the web site should be friends.
One app likes to manage state,
The other site’s less than 48 (kB),
But that’s no reason why they cain’t be friends.

Clearly I shouldn’t plan on switching careers to go write a musical any time soon. 😅

All kidding aside, a debate continues to rage in the broader #webdev industry, and that is the question of what constitutes a web “site” vs. a web “app” and why the development techniques used for one vs. the other can vary so dramatically. It’s even been suggested that the web essentially “bifurcate” into two different (but loosely-coupled?) platforms. The document web as some folks call it would have one set of considerations and constraints, and the application web would have yet another.

A Clean Start for the Web by Tom MacWright is probably the best articulation of this concept. Tom’s also written a couple of seminal articles about web architecture in general such as Second-guessing the modern web and If not SPAs, What?, so it’s well worth your time to read through those if you haven’t already.

I have many thoughts on this matter (surprise surprise 😉), but I’ll elaborate on just a few key points which I hope may help us make sense of this and a possible path forward.

Point #1: The Dearth of Novel HTML-First Features Has Been Appalling #

As much as I love custom elements and as much as I realize there are already quite a few tags in HTML5 that everyone including myself tends to forget about…the sad fact of the matter is that HTML still feels like the weird uncle living in the attic compared to the bling-bling-wearing celebrities that are modern CSS and ES2022+ JavaScript.

How many articles, podcasts, presentations, and educational tracks are you familar with centered around JS? CSS? Now how many do you know of which are centered around HTML? 😂

I think this simultaneously causes two problems:

• Document Web people feel abandoned, like they need a higher-ed degree in React / TypeScript / eslint / esbuild / esthingamajig to get anything done because all new web APIs seem to be overwhelmingly JavaScript-centric. HTML’s pace of development feels positively glacial by comparison. HTML can’t even construct itself out of more than one file (aka HTML imports or modules or whatever you want to call it), yet it’s a technology that’s over 30 years old! WTF 😠
• Application Web people continue to feel smug in their ivory tower of abstractions, regarding HTML as little more than a “build target”. In fact, from that vantage point, the closer the markup can get to compressed bytecode the better. I’ve legit had web developers tell me they wish they could just ditch HTML entirely. WTF 😠

It might sound like I’m biased on the document web side, and perhaps I am, but that’s not really it. I’m biased on the HTML side. So by HTML lagging so dreadfully compared to the other technologies in the web spec, it seems to reinforce the notion that the document web has been sidelined in favor of the application web. And because of the anemic nature of HTML’s progress over the lifetime of the web, it subtly reinforces the notion that to be a fancy-pants web application developer, your knowledge of HTML doesn’t need to go much past <div>/<span> and class= (or would that be className=? …shudder), and you’re better off keeping your focus on higher-level JavaScript abstractions.

I don’t know what the solution is to this, but I think for a variety of reasons a renewed effort to modernize HTML and port more imperative JavaScript APIs to the world of declarative markup would go a long, long way in (a) placating the document web folks, and (b) encouraging the application web folks to bump HTML up in priority and prominence. Thankfully, because we do have custom elements and adding custom attributes onto existing tags is easy-peasy, we have the ability to prototype the APIs we want without having to wait for the spec to emerge. So get cracking, people! I want to see your beautiful declarative HTML APIs.

Point #2: The Progressive Unfolding of Advanced Functionality Will Be Supported by Next-Generation Tooling #

We’ve all heard the term progressive enhancement (PE) a bazillion times by now, and the truth of the matter is it sounds nice in theory but rarely seems to pans out in practice. I doubt there are very many forms you can submit or interactions you can make in app-like UIs without JavaScript enabled anywhere on the modern web. And while the number of brute-force hacks we have to endure in markup simply due to styling concerns has decreased, we’ve also seen the rise of atomic CSS and abstracted UI frameworks and so forth which quickly renders your HTML looking like <div> tag soup.

So the old PE canard of “First, write your semantic HTML and verify the structure of your content/UI. Then, write your stylesheet and verify the appearance of your content/UI. Then, write your JavaScript and verify the behavior of your content/UI. And make sure the most critical functionality works for the largest number of user agents even in adverse conditions” seems wildly at odds with how many web developers work today, especially in the world of single-file components (SFCs) and meta-frameworks.

I still feel like PE should be a valid methodology in our world, but somehow the marketing needs to change, because we’re just not winning this battle. Maybe there’s a way to view this not so much as a step-by-step methodology during development, but more as a spectrum of tooling and practices which will vary from project to project or even within a project from feature to feature.

In other words, to go back to the original premise, what if we could use “document web” tooling/practices on this page here and also “application web” tooling/practices on that screen there? What if the level of functionality, or fidelity, or whatever you want to call it, could be dialed up and down as needed without throwing the entire toolchain out? For example, right now if you decide you’re going to build a website with Next.js, you’re immediately locked into a huge set of assumptions. React everywhere. SPA-style routing everywhere. Webpack chunks streaming in via JavaScript everywhere. Huge piles of state data for re-hydration. And so on and so forth. (Although in the future, Next may look quite different in that regard.)

I feel like the “meta-framework” of the future will start with a totally different set of assumptions. Perhaps Astro is really onto something here. In other words, you’d start quite literally with a “document web” mindset. Simple HTML pages and formats like Markdown. Zero JavaScript. No framework lock-ins. Minimalist tooling in general. And then slowly, gradually, deliberately, with mindfulness, on a page-by-page or even component-by-component basis, you could increasingly go nuts. Load in all the frameworks and state management and form builders and whatever you need—but still contained within those particular areas and ideally “lazy loaded” based on user intent. Same principles would work on the server-side: HTML that initially comes from static builds by default, but then you could opt-into SSR wherever needed for real-time content updates or processing form input or whatever. (Next does offer this already, but still you’re locked into using complex client-side tech such as React to render…checks notes…HTML on the back-end…just feels too often like you’re getting rid of ants by firing bazookas).

Some are starting to call this approach islands architecture, and I really like that nomenclature. You start with the ocean, calm and sirene on the surface, and then feature-by-feature you define your islands of high-fidelity interactivity and code complexity within the vastness of the ocean.

Point #3: The Progressive Unfolding of Advanced Tooling Will Be Supported by Next-Generation Functionality #

See what I did there? 😜

As an important analogous programming concept to the one we just covered, I’m also highly recommending gradual typing. (Hat tip to Stephen Hagemann for his very thoughtful response to my recent Spicy Web article on static types.) Let’s throw gradual linting and gradual testing into the mix while we’re at it. 😱 Too often we’ve seen industry “best practices” suggest using heavy-duty developer tooling such as TypeScript, eslint, TDD frameworks, etc. for everything in an entire web project from Day 1. I really chafe at such a dictatorial predilection. (Say that ten times fast!)

What if we could be smarter about configuring our tools so that it’s easy to slowly, gradually, deliberately, with mindfulness, on a feature-by-feature basis, adopt static typing, linting, tests, etc. I truly don’t consider it a bug that you can mangle HTML (and even CSS) quite a bit and it still sorta-kinda works. I consider that a feature. JavaScript is less forgiving but it’s still pretty flexible out-of-the-box.

This is what has made the web so outrageously successful for such a long period of time. Accessibility isn’t just a noble goal to check off for the public using a web page. Developer accessibility is also vastly underrated as a vital aspect of web development. The bar for entry to becoming a web developer must remain reasonbly low. As it stands, it’s become too damn high. (I’ll also throw out the Right to Inspect concept here, but that’s an article for another time!)

We shouldn’t have to debate enterprise-grade code/advanced tooling vs. fast code/minimal tooling as a binary yes/no consideration project-wide (much less company-wide!). Our environments should be sophisticated enough that we can opt-in however and whenever we need to. And to circle back around, our web toolkits and frameworks similarly should start at the most base level of simplicity possible, and then “evolve” in complexity—not “over time as the web industry grows” which has typically been the case (hence the mind-numbing amount of complexity we see today), but over our own time as we work on projects and features.

Bottom Line: We Just Need Smarter Tools, So We Can Be Dumber #

To sum up everything in some fashion, what I’m driving at is we don’t need a separate document web and application web, but we do need our tools to get a whole lot smarter so that we can easily pop back and forth from doing “document stuff” to doing “application stuff” and back again without breaking a sweat—as well as support a whole range of skill levels and performance/quality/maintainability trade-offs in-between.

As a clear way to illustrate this issue of tooling (and apologies for the blatant shilling of my open source project): before I started working on Bridgetown, a progressive site generator powered by Ruby, I built quite a few client projects using both Jekyll and Rails. Jekyll for the public-facing static site needed by an organization, and Rails for additional dynamic functionality (forms, payments) and content management and so forth.

Due to Jekyll & Rails both being Ruby-based software, you’d think they’d get along quite nicely in a monorepo and complement each other well, right? LOL. 😂 They often felt worlds apart, and when using Jekyll I’d miss all kinds of features and methodologies and mindsets which are implicit in Rails and server-side webapps, and when using Rails I’d miss all kinds of features and methodologies and mindsets which are implicit in Jekyll and statically-generated websites. You couldn’t even deploy them similarly! One would go off to Netlify or whatever to get statically-rendered, the other would go off to Heroku or the like for a fancy-pants database-driven installation. Wiring them up as if they were a single atomic “deployment” was a nightmare.

In other words, it was the “document web” and the “application web” in stark relief.

Fast forward to today. My over-arching goal as I work on Bridgetown and subsequently work on downstream projects using Bridgetown is to bring the full spectrum of the document web and the application web together in a single tool and truly offer that “opt-in only when and where you need it” developer experience. I have no hesitation in stating that it’s dramatically easier to get started using Bridgetown than Rails for a content-centric project. And I have no intention of stopping there. Obviously if your application needs are quite advanced, Rails is going to meet those needs far better than Bridgetown and the Roda web toolkit it melds with for SSR. But over time, I expect an increasing number of site/app hybrid projects will fall right within Bridgetown’s sweet spot.

Or maybe Astro’s.

Or maybe Rocket’s.

Or…

I think vaguely we all know where we need to head. It might just take a little while longer to get there.

P.S. Yet another analogous concept along these lines is the idea of the self-provisioning runtime…you wouldn’t even need to know how best to deploy your particular site/app! Just point your service provider at your repo, and it figures it all out. Static site? Check. Server containers? Check. Serverless functions? Check. Database? Check. Key/val store for caches? Check. Background jobs? Check. Global infrastructure at key times in certain ways to meet demand and decrease latency? Check. After all, why should you, O Human, need to know all those things? Shouldn’t the computers be smart enough to know how best to boot up and run their computer-y things? (The answer is yes. Obviously it’s a very hard problem to solve. But solve it some people will, I have no doubt.)