About a decade ago, I tried to index the web via the Dewey Decimal System. I had a site laid out similar to Google, where you could browse sites continuously starting from a given call, but the DDS is proprietary, and those people hate anyone who uses their IP without a license. You can Google everyone they’ve shutdown – places that weren’t even libraries – for using anything similar to the DDS. I reached out to the group that manages the DDS, and was taken offline before my project even started.
With all the corporate BS lately, and people looking for alternative options, I thought I’d take a search engine old school, and we’d index like Usenet. Except with an XXX.XXX.XXX format.
My original project used sharded Redis, with append logs to disk. I chose it for its in-memory speed and key-value store. Did some calculations, and I’d have to have millions of records just to consume my entire system’s memory.
I had a lot of plans for this before getting shut down.
Now that I’m older, I’m curious if I should be using MongoDB.
What are the benefits and drawbacks of each? Which would you use? And why?
And not just thinking, it actively imposes an architecture onto your codebase. It pretty much forces a CRUD API, which only really works well for a client-server structure. And it forces you to break up your data structures and introduce IDs to a degree that you would simply not do while programming normally.
I mean, I can see the appeal. If these constraints are fine for you and you don’t have other constraints, like sparse data, then building your whole application on top of SQL gives you clear answers for how to architect that.
But yeah, I also really resent this idea that it should be the default, because when it does not match the problem domain, you spend a lot of time working around the architecture that it imposes.
You can use SQL however you want
You don’t have to normalize your data. In fact, denormalization based on access patterns is a common optimization technique.