Start With the Domain

So you have finally started working on this greenfield project! Or perhaps you just joined a new company or team? Maybe you got an assignment at university or you finally decided to give that startup idea a go.

You’re starting a new software project.

But instead of setting up the repository, buying the domain, picking the stack and sweating the technical details, you need to do something else first.

To the disappointment of many engineers, you need to understand the business and the product you’re setting out to build.

A Quick Aside

This is not just an article full of code examples, it’s a chapter of my upcoming book “The Full-Stack Tao” which I’m writing in public.

Here are all published chapters so far:

The Sweet Spot in the Middle

Thinking about the technology without understanding the domain it will exist in is similar to deciding on a vehicle to travel with without knowing the destination.

You may pick a stack that leads to an over-engineered solution and stall the development of an otherwise simple product. Or you might use a technology that’s far too limited for what you’re about to build.

No matter if you’re under or over-engineering, the result will be the same - subpar software.

A big part of our job as engineers is to continuously push the product we’re building to that sweet spot in the middle. There we’re capable of both shipping at a productive speed and are confident in our stack’s capabilities.

But we can’t find that balance unless we understand the business we’re supporting.

What if You Don’t Learn the Domain?

Most engineers have a knee-jerk reaction when it comes to learning business details. It’s someone else’s job after all. Product and marketing people can take care of that while we sweat the technical details.

So instead of convincing you, let’s explore what happens when we don’t understand our domain.

We develop highly-scalable systems for users that are in the low thousands. We design our data to fit into NoSQL stores even though our traffic would hardly push the limits of a low-tier MySQL database.

We create abstractions upon abstractions so we’re insured against imaginary scenarios. We build monoliths, putting components that could exist independently in the same place.

We create design systems without the capability to support them. We obsess over unnecessary re-renders while our performance bottlenecks lie elsewhere.

We invest crazy amounts of time engineering solutions that hardly bring any benefit. We pick technologies based on our tastes, then struggle to shoehorn every problem in a way they can solve.

I’m sure you’ve been in at least one of these situations. The common denominator between all of them is one - lack of knowledge about the product.

And that leads to bad technical decisions.

The Domain Dictates Everything

A product-savvy engineer is more likely to find the correct level of technical complexity for the product they’re trying to build. And that’s far more impactful than choosing between React and Angular for the UI.

Everything from scale to architecture and libraries is dictated by the domain.

The level of scalability you need depends on the traffic you expect. The level of elasticity depends on whether the business will have sudden spikes in traffic. Even the depth of your abstractions and your data structures will depend on the expectation of the product.

A few years ago, I worked on the implementation of a rule engine. And even though we managed to create a perfectly sound implementation using a graph, in the end we moved to a solution using multiple trees instead.

The tree-traversal algorithms made more sense for the reality of the business, even though the graph was the textbook solution.

Best Practices

Best practices are a good starting point when you’re considering a solution. They’re generic guidelines that are applicable in the majority of cases. They give us general advice, but sooner or later every software diverges from them.

Once you get into the details you will find that every business has specific needs that are hard to address with generic knowledge.

Being aware of the nuances of the business model will help you make informed decisions about which practices to follow and which to discard.

Looking into the Future

Nowadays no software is set in stone. Things will change.

But understanding the domain helps us ground many of the common fears and uncertainties related to architecture and even low-level coding decisions.

Worrying whether you will have to change the database only makes sense if there’s a realistic expectation of growth or a shift in the business. Evaluating if a library has enough features to support you in the future takes less time if you have an idea of what they could be.

Looking into the future of a software is looking into the forecast of a company. And it’s only possible if you’re familiar with the domain.

Unclear Requirements

And because these seem like problems of unclear requirements, we may quickly shift the blame to the business people again. It’s hard to make the right calls when you don’t know what people want.

But here we get to the root of the problem - knowing what people want.

Understanding the business will help you ask the right questions. You will be able to identify edge cases and potential problems before they’re implemented and discovered by a user.

Product people won’t be able to identify the technical gaps in a feature, but with some domain knowledge, you will.

Imagine a software company developing an online reservation system for a chain of boutique hotels. The directions you get are that customers should be able to book rooms online.

The development team, taking this at face value, creates a simple system where customers can select a room and book it for a specific date. The system is implemented, and on the surface, everything seems to work fine.

However, once the system goes live, problems arise. Customers are accidentally booking rooms that have already been reserved. Some are booking rooms that are under maintenance. There’s no way to handle special requests or discounts for longer stays.

If the development team had a deeper understanding of the hotel business, they might have asked questions like:

How are room availabilities managed?
How are rooms that are under maintenance or renovation handled?
Is there a need for a feature that allows special requests, such as a crib for a baby or accessibility features?
Are there any discounts or special packages for longer stays or special occasions?

You may think that most developers would’ve been aware of these details. But that’s only because we’re all deeply familiar with how hotels work.

Imagine a similar situation but in the finance domain.

Eliminate Complexity on the Product Level

I had a project in which we used an external auth provider for authentication, but we still wanted to create an entry in our database whenever a new user signed up. We would then use the data we have stored in our database for certain relations.

But this creates a gnarly race-condition.

When the user authenticates with the auth provider and enters the application we may still not have created their account. Even worse, there might have been an error or a temporary blip that would leave their account in an unusable condition.

This led us to think about very complex synchronization mechanisms, empty UI states and various doomsday scenarios we had to take care of.

But instead of handling all that, we decided to add an extra screen in the UI that would prompt the user to submit their additional details so we can store them in our database. This way we eliminated the creation of an entire distributed system.

Understanding the product will help you find ways to eliminate complexity before it becomes a technical problem.

Of course, in some cases that is not possible. But if you limit the unnecessary complexity, the necessary one will be far easier to accommodate.

How to Learn the Domain?

The average software engineer changes jobs every couple of years, and they have to learn a new domain every time - it’s rough.

The good news is that you don’t have to become an expert on day one. Here are some questions that I like to ask in order to expand my knowledge about the business.

1. Who buys the product and who uses it?

Often the buyers and the users are two different groups. This will show you why people use this product and what’s important to them.

2. How do they use it? Is it sitting in an open tab all day? Are they using it in small windows during their commute? Is it critical to their life and work?

Long-lived state management is a problem for most front-end applications. But most media companies have found a way to avoid it. Statistics show that most of their users browse their websites atomically. They open a page and they leave, or they open a couple of pages then close the website. The session duration is quite short.

Because of that many medias have opted to use server-side routing for all their websites. This way they don’t have to manage state in the client, and they can rely on CDN-level caching.

3. How many users does the product have?

Gives you a sense of the business’s scale.

4. What are the future plans for the part of the product you’re working on, no matter how big or small?

If the company is planning a marketing push to acquire new users, this puts scalability much higher in the priority list. If you’ll be building a tool that they’re planning to white label and sell to other companies, this means you will need an abstraction over how styles are applied in your React components.

5. But the thing that helps most of all is being a user of the product yourself.

If you’re building something for end-users, make sure you try it out as well. If you’re building an enterprise software, get someone to walk you through it.

Answer all these questions, play around with the product a little bit and you’ll see that the architecture is making itself. You will know what constraints you have to stick to, what amounts of data come in, how quickly the UI should become resopnsive, what devices it should be viewed on, and so on.

But if you’re working on a very small slice of this product, do you need to be aware of everything? Can’t you just learn your niche of the domain?

Want to Learn More?

This is a chapter of the Full-Stack Tao book that I’m writing right now. I will continue to share the most important free chapters on my blog and through my newsletter which is linked below.