The significance of natural language processing\n\nNatural language processing (NLP) is currently gaining and maintaining widespread attention, generally in relation to its significance to the continued development of ever more advanced artificial intelligence, or AI.\n\nThis is for good reason; historically, natural language, although easily understood by human beings, has been an unprecedented navigational challenge for computers. However, although the degree is up for debate, there is no question that advancements in NLP are changing this latter fact, and computers are ever-closer to being able to effectively understand and manage natural language fruitfully.\n\nWhat this will eventually lead to is computers that are able to understand not only structured data in the form of programming language, but also to understand data in the form of natural language\u2014both structured and unstructured. It is precisely this impact on AI that is the most revolutionary.\n\nThe data-information-knowledge-wisdom model\n\nThe significance of the current advancements in NLP\u2014especially in relation to unstructured data\u2014becomes apparent in light of the DIKW Model (often represented in literature by a pyramid), which I have discussed in the past (including specific considerations related to the knowledge -> wisdom peak of the model). The model highlights, among other things, how data and information lead to knowledge, and how knowledge \u2013 managed well \u2013 leads to organizational wisdom.\n\nWhat I like about the model is that it also visually represents key relationships and phenomena, one specific example being what I call a \u2018knowledge failure\u2019; these are failures to efficiently and effectively manage knowledge to create organizational wisdom (as part of the knowledge -> wisdom peak.)\n\nKnowledge overproduction as knowledge failure\n\n\u2018Knowledge overproduction\u2019, one such type of failure, is defined as \u201cthe development of knowledge in excess of that required to optimally maximize organizational wisdom\u201d. At least, this has been the case historically. A corollary to this is that the challenge for any organization is not to always know more, but instead to know enough.\n\nThis is because there are downsides to production of knowledge beyond that which is necessary. I have pointed to the downsides of the overproduction of knowledge: most notably are wasted efforts making sense of things (I am a fan of David Shenk\u2019s visual reference of \u2018data smog\u2019). But also, to lost effort directed at knowledge development, sharing, processing, and management (because underlying knowledge creation is effort towards obtaining data and information, and further developing information into knowledge.)\n\nHowever, with current advancements in NLP, the very nature of knowledge overproduction as a \u2018knowledge failure\u2019 is changing.\n\nKnowledge overproduction and natural language processing\n\nThe anticipated ability of computer programs to understand not only programming language, but natural language, brings to the field of knowledge management a fundamentally qualitative leap. The overwhelming amount of data and information in the world falls under the spectrum of natural language. This is where NLP holds its potential.\n\nMake no mistake, the field of computing has already brought powerful abilities to bear to knowledge management. This is true at the fundamental level of computing itself. However, historically, efficient and effective management of knowledge has been limited by the individual human beings involved (albeit with the support of software and hardware). But the extent to which those powers shift with NLP is monumental. \n\nWhat NLP does is make possible the efficient and effective management of inordinate amounts of data and information, beyond what has ever been possible by human capacity. And it would make this, for all intents and purposes, effortless.\n\nKnowledge overproduction: an opportunity\n\nWhat occurs when a non-human entity understands natural language to a sufficient degree to efficiently, effectively, and almost effortlessly manage limitless knowledge?\n\nAnother view of maximizing the knowledge -> wisdom transformation process is to look not at \u2018knowledge failures\u2019, but \u2018knowledge opportunities.\u2019 Juxtaposed to knowledge failures, these are opportunities to more efficiently and effectively manage knowledge to create organizational wisdom. With its ability to manage unstructured data at a potentially unlimited scale, NLP is contributing (together with secondary factors such as digitization and big data), to the greatest knowledge opportunity of this generation.\n\nAs a professional in the health care industry, just one example of unstructured data interaction is that of specialists are employed to review clinical documents on a regular basis. Given that this is frequently unstructured data, a human being historically reviews it for a stated purpose. And this is true not only in hospitals, but also in pharmacies and other health care settings. As just one example, imagine if an organization could quickly have its finger on the pulse of the aggregate and component data related to the minute details of things as significant as long-term patient outcomes. And this is but a speck of the market.\n\nImagine all of the data in the current era of big data: emails, reports, publications, texts\u2026the list is as endless as technology is relentless in pushing modes of producing, storing, and sharing knowledge. And imagine this across the industries of today \u2013 banking and finance, energy, information technology, social services\u2026all of it, with tremendous costs to production and refinement. And historically, in consumption and synthesis.\n\nImagine AI, with natural language understanding (NLU), radically changing the very foundations of the feasibility and cost structure of the review of unstructured data. NLP has an ability to radically shift how we transform knowledge into organizational wisdom. Eventually, NLP could bring us full human language in all digital systems, with almost effortless knowledge management and synthesis.