In this post…
Do you know that feeling when you see that a colleague is doing something not right, you care enough to want to speak up, but are also insecure and think that they might know better than you? Yes, me too.
Today I’m going to share another of my technical debt lessons, which is something that happened to me about a year ago, after I started working on this project which had already been scaffolded and boiler-plated. Up until then, I had always worked with relational databases. Oh, the beauty of a relational table with lots of ids!
This project, however, had been started in NoSQL, which was new to me. I had to start thinking about “collections”. What was wrong with the good old “table”? Anyway, I approached this how I usually approach any new technology: with curiosity, keenness to learn, and lots of research. One of the first things that I did was go to the MongoDB website and start a tutorial to get familiar with the structure, namings, query structure, and so on.
Then, I dug in and started creating some reports in the application.
That’s when I noticed something was off… There were quite a few “ids” in the collections. Now, if you know NoSQL (and, if you don’t, treat this as a brief introduction!), you’ll be aware that the paradigm is totally different than with SQL.
Allow me a little break to introduce three differences between SQL and NoSQL, which you need to know to follow this story.
A quick SQL vs NoSQL comparison
SQL
- Data is stored in tables
- We normalise at least until 3rd Normal Form
- This means that, in a table, we avoid repeating data that exists elsewhere, and simply reference it (by using the id of that field)
- Think of an order made by a user. We wouldn’t store the user’s phone number with the order, as it might change! So, instead, we’d store the userId, because the phone number is associated with a user
- This means that, in a table, we avoid repeating data that exists elsewhere, and simply reference it (by using the id of that field)
- Data structure is rigid
- This means that you need to define the table columns beforehand
NoSQL
- Data is stored in collections
- We purposefully denormalise
- This means that, in a collection, we repeat all the data that is likely to be used together (by using its value)
- In the same example, we would store the actual user’s phone number with the order, if that was to be reported together
- This means that, in a collection, we repeat all the data that is likely to be used together (by using its value)
- Data structure is fluid
- This means that you can define the collection parameters “on the fly”
Break over, now back to the story!
As I started doing these reports, my alarm bells started going off… Using all those “ids” to reference other tables was wrong in a NoSQL paradigm, and it was forcing me to join seven different collections at a time. In SQL, this is a headache, but “it is what it is”. In NoSQL, this is a design smell.
It wasn’t a matter of taste or style anymore, it was a structural decision that would shape the system long after we moved on. At that point, the technical problem had turned into a human one: who owns a bad decision if everyone sees it and no one says anything?
We were a team of two (said colleague and me). Even though we were on the same level, my colleague was a bit more senior than me. He usually handled the architecture and I focused more on the UX and frontend, ensuring that the software stayed intuitive, all edge cases were accounted for and, in general, the end product looked polished.
I didn’t want to be “that” person.
I was afraid of overstepping.
I didn’t want to create friction with the person that sat next to me every day.
I thought that I was probably missing something obvious. So I wrote impossible queries in MongoDB to get the data that I needed. It wasn’t pretty by any means.
But the more research I did into NoSQL, the more obvious it became that the data model was working against us.
So I started mapping out what a better model should look like. Once this was ready, I had to make a decision. Paraphrasing William Shakespeare:
To speak up or not to speak up, that’s the question
I was afraid, but I decided to speak up for the sake of the project, the language, and whoever had to maintain that in a couple of years’ time.
I asked for a meeting with him and our manager. While I was walking there, I was sweating, my heart was beating fast, and I was nervous. What if I plainly showcased myself as someone who didn’t know what she was talking about? What if I made a fool of myself? All these doubts were nagging me at the back of my head.
During the meeting, I explained the issue, showed the outrageous query I had had to write, and proposed a solution. They looked at it and agreed. We acted on those changes the same day.
The reports were greatly simplified, as should have been from the beginning. The database was used as intended. The loading speed increased. Birds would be happily humming, if there were some around.
Summary
The socially-easy choice, but technically hard
- Avoid the meeting
- Force the tech to fit the problem
- Result: “impossible” queries and nightmare maintenance
The socially-difficult choice, but technically easy
- Have the awkward conversation
- Fix the data structure at the source
- Result: simpler queries and a scalable system
What did I learn from this experience?
Good engineering isn’t silent obedience or “staying in your lane.”
It is taking responsibility for the system you’re building together.
It’s caring enough to question decisions, respectfully and with evidence, especially when staying quiet would cost the whole team in the long run.
Opting for the flexibility of NoSQL often feels like a ‘hack’ for productivity, but without a plan for the data’s evolution, it quickly becomes a high-interest loan.
It reminds me that our systems are built by people, for people, and the debt we accrue is always more than just syntax.
A question for you
Have you ever stayed in your lane even when you saw a collision coming? What stopped you from speaking up?
Leave a Reply