In my last blog post, I talked about the emergence of NoSQL as an antidote to the deficiencies of traditional SQL RDBMS products, and I concluded with the question about where the data management industry is going given the current environment, and whether we\u2019re really addressing the needs of senior technology leadership.\nLet\u2019s start with going a bit further into the trade-offs represented by NoSQL.\nKISS and the cloud \nThe NoSQL movement is not merely a slam against the traditional RDBMS. NoSQL seeks to offer solutions, solutions that address the list of needs I outlined in my last post.\nThe central design theme of NoSQL database systems is semantic simplification. \u00a0Many of the following things are finding their way back into the systems today, but early NoSQL systems pursued the cloud-style requirements by avoiding schema design, normalization, consistency guarantees, and server-side processing complexity. \u00a0The idea is to trade powerful traditional database capabilities for powerful cloud-style database capabilities. \u00a0\nSimplifying a database system is the easy route to elastic scalability. \u00a0File systems scale arbitrarily because they are simple, independent key-value stores with very few guarantees of anything. What Jack does on his laptop has nothing to do with what Jane does on hers. Obtaining more laptops linearly increases storage capacity, I\/O throughput, and user concurrency. It is the complexities (such as maintaining files consistently across machines or maintaining formal relationships between files on different machines) that would limit the scalability of the overall system.\nThe easy way to autoscale is to trade semantic power of database services for elastic scalability. \u00a0\nBetween Scylla and Charybdis \nIn this new world, the CIO is presented with two choices:\n\nStick with traditional database technologies, with all of the proven power of those systems but with limitations on elastic scalability (see my post on database elasticity as the primary challenge for digital transformation), or\nMove to a semantically simpler data management solution that scales elastically, and accept that it is more limited in richness of services.\n\nThe SQL RDBMS is the gold standard of enterprise data management. Organizations have established toolchains, practitioners, operational models, security policies, business continuity processes, data integration and warehousing strategies, and vendor relationships. Above all, the systems deliver trusted transaction models and standard data models that decouple strategic corporate data from the applications that interact with it. \u00a0\nOn the other hand, moving away from the SQL RDBMS can lead to important benefits in terms of cloud-style deployment. Doing so involves a significant institutional learning curve, business risk, and a much more limited ecosystem. There is a smaller pool of application development skills, and reduced database SLAs (specifically ACID transactions) imply increased application complexity, skills scarcity and lifecycle cost.\nAs relates to this choice, CIOs have had to be pragmatic: The common approach is to stick to RDBMS-based solutions if at all possible, even on the cloud, and adopt more cloud-friendly solutions in situations in which the RDBMS falls short. The latter situation is increasingly the case as elastic cloud deployment drives strategic direction.\nConservation of intelligence \nAt a system architecture level, this choice between the traditional RDBMS and the non-SQL solutions is a choice between smart databases and smart applications. \u00a0In simple terms: Where do you want to put the intelligence?\nThere are two architectural elements to the question: SQL and ACID transactions. One enables powerful server-side processing, and the other provides a simple model of data guarantees that enormously simplifies applications. \u00a0\nA central question in relation to data management is whether you want a server-side language. Without one, your application has to grab lots of data across the network and do all non-trivial data manipulation on the client side. The benefit is that the server is simple and may be able to scale-easily. The downside is that the client has to do much more work, which is costly, error-prone, and probably duplicates what other clients are doing. There are also negative consequences relating to latency and consistency\/concurrency, though these are a little less obvious.\nA server-side language can enable a smart server that does powerful processing on behalf of clients. Likely benefits include simplification of client code, as well as performance of non-trivial data manipulation processes. Parenthetically, it might be noted that there are good arguments for the server-side language to be a) independent of application languages, b) declarative in nature, and c) set-based \u2013 in other words something pretty close to SQL. The downside of a smart server is that it is more complex, and furthermore the complexity could limit other attractive features (like elastic scalability).\nThe other architectural element relating to client-side vs. server-side intelligence is ACID transactions. You sometimes hear people say that ACID transactions are only really necessary for a small subset of applications, but that assertion misses the point. \u00a0It is not about whether transactions are necessary or not \u2013 it is about application simplification. \u00a0\nWhen a database system provides formal guarantees of Durability (the \u2018D\u2019 in ACID), for example, an application does not have to deal with exceptions relating to the data perhaps not being committed to durable storage. \u00a0Analogous, and more serious, observations apply to the other dimensions of ACID (Atomicity, Consistency and Isolation).\nApplications built on SQL\/ACID services can be simpler, cheaper and more reliable, and developed by lower-skilled teams. But there are times when this is a poor trade-off. \u00a0Cloud-style infrastructure is the future, and if application complexity is an unavoidable cost then it may make sense to pay it.\nYou might ask whether there is a way to have both. Can we have smart databases that can scale on the cloud? \u00a0\nIn my next post, I\u2019ll tell you why I think the answer is yes. But in the meantime, what do you think?