CIOs must consider the costs of managing Linux and open source licences

Open source software has become a significant component of all software development activities, intentionally and sometimes unintentionally, thanks to the wealth of available source code, its apparent free cost, and high degree of stability and security. But while open source appears to be cost free, it is not without obligations, as it comes laden with licensing and copyright responsibilities enforceable by law. Lack of knowledge about these obligations and ignoring them can lead to dire consequences for technology firms, and some of the ensuing legal cases have been well documented.

This does not mean that outsourcing or open source usage should be avoided. The cause for concern is not with the use of open source, but with the unmanaged adoption of third party code and its accompanying copyright and licensing duties. It is important for software organisations to establish appropriate intellectual property (IP) policies that determine what specific open source licenses and license terms are acceptable for a specific product and business. Managers need to validate the IP cleanliness of their products and services to make sure all legal obligations are met before they go to market.

A cost model for assuring software legal compliance is essential for CIOs.  A cost model must take into account factors such as the extent of open source or other third party content in a product, the extent to which the content violates an organisation’s licensing policies, and the probability of detecting a violation after a product launch and the cost associated with fixing the problem. Considering a number of scenarios with varying project complexity, organisation size, and introducing cost numbers for correcting licensing violations during development, this model provides a glimpse of the effectiveness and the economies of automated software scanning and licensing compliance.

Third Party Software Content and IP Violations

Nowadays, it’s common to have software products consisting of thousands of software files (source code or binaries). Some of the components brought into the product may have license requirements and copyright obligations that are at variance with the corporate IP policy. For example a corporation may be legally compelled to release the source code for a commercial product, creating a serious loss in revenue for the company. On the other hand, another scenario involves modification of a software file while the license specifically forbids any tampering with the code, resulting in legal action. Some of these violations are significant enough to warrant specific copyright compliance actions or software corrections, although determination of what is a significant legal violation is ultimately for a judge to decide; and getting in front of a judge is an expensive proposition.

The level of external content in the software could be as little as 10 per cent and as high as 100 per cent if the software component of the product is completely outsourced. For an illustration of our cost model, let’s assume that only 45 per cent of the software components are open source, or otherwise of external origin.  In this model we shall assume that only four per cent of all external content is in violation of the associated corporate IP policy.  If a licensing violation is detected after the software is released to the market, then costly post-release corrections are necessary. The model presented here allows for a range of non-compliance visibility in the market. Here we will assume that about 15 per cent of the violations are some how detected and reported in the field.  In other words, 85 per cent of IP policy violations remain unnoticed in the field and cause no problems (until they may be discovered, of course).

Methods Deployed For Software Licensing Compliance Traditionally, assessing IP-cleanliness has been done manually, by relying on developer records, expert analysis of the final software, and due diligence processes. The licensing and copyright assessment is mostly undertaken in advance of important financial transactions – an investment, a merger/acquisition, or an impending product release. Manual assessments are prone to error, consume expert resources, take a long time and are becoming quite expensive nowadays as the use of open source and outsourcing grows.

Mitigating business risks associated with software legal compliance is best addressed by adopting a process, including legal considerations, within an organisation’s software quality development process. The following options are available to organisations, and address compliance measures at different points in the development process.

• ”Head-in-the-Sand” Do Nothing:Popular up until recently, this option ignores the compliance issue because it carries the lowest up-front cost, but bears the highest business risks and largest corrective costs post market introduction.
• Developer Training and Project Planning:Some companies consider that proper training and project planning is sufficient in normal situations, accepting to undertake an audit during imposed due-diligence efforts. Naturally, the more the developers are trained on matters of software legal compliance issues, the more effective the development process can be. This is, however, a rather expensive proposition, given the growth in number of distinct software licenses, the cost of developer training, and the constant churn within the development environment. With this option, compliance depends solely on developers and there is still no assurance of legal compliance before going to market.
• Post Development Licensing Analysis and Correction:Taking action later in the project lifecycle can take the form of external or internal auditing, and impacts the final stages of testing and the quality assurance process. This option can become expensive due to any necessary changes to the software after licensing analysis, subsequent re-testing and re-assessment. However, this option does not impact development workflow, and can be rendered more cost effective with software tools designed for this purpose. It can nevertheless prolong the project lifecycle near the end, resulting in unpredictable delays to the delivery of the final product.

• Periodic Analysis and Correction:Periodic licensing analysis and assessment of software during development leads to corrections along the way if IP policy violations are detected. The analysis can be done with automatic tools, and is less expensive than assessment the conclusion of development process, thanks to shorter delays in getting the fixes done and re-tested.

• Real-time Preventive Assistance at the Developer Workstation:The most pro-active measure for software licensing compliance is to detect license violations immediately at the developer workstation in real-time.  The development process is not disturbed, and the cost of corrections is minimized as any necessary corrections are done on the spot without involvement of other resources and without need for re-testing.  This process can be automated via software tools in ways that are unobtrusive, easy to adopt and, most importantly, do not require developer training in matters of legal compliance.  Managing licensing or copyright in real-time is the most cost efficient and lowest risk option in the long term.

Some of these options, such as real-time and periodic or build-stage assessment, can be used in combination for better results. Generally the sooner problems are detected and fixed, the lower the cost of the licensing management would be. The efforts to fix the software IP issues and the associated delays in product readiness will drive the economics of software licensing management.

Effectiveness of Automated Software Scanning and Licensing Management Tools

Fortunately, there are tools available to automatically scan software (source code or binaries) and conduct a software pedigree analyses, detecting all licensing and copyright policy violations.  These tools can operate on demand, on schedule or even in real-time within the development process.

Some of the automated software scanning solutions allow the software analyses to be done in accordance with corporate IP policies and lend themselves well to instituting proper record keeping and safe software development practices.

Most software IP scanning and licensing analysis tools have a performance-correctness factor between 80 per cent and 98 per cent, where performance is negatively affected by leaving-out IP policy violations or reporting false positives (i.e. false assertions of violations). This performance depends on the accuracy of the analysis engine and the size of the external software (including open source) database used for reference. For illustration, our model will assume a degree of performance for the corrective analysis tool of 95 per cent. An automated, preventive, licensing management tool has a higher degree of performance, as it scans and detects every new software component which is saved/filed at the developer workstation. Its ability to detect external content is close to 100 per cent, but for a conservative analysis we shall assume a 98 per cent performance correctness.

Costs to Detect and Fix IP Policy Violations

The worst case is to have license or copyright violations discovered in the field, or during an audit prior to a major financial event. In such cases the costs are much higher due to involvement of legal personnel and the corrections necessary after development completion. Not taking into account having to face a judge, the costs can be anywhere between $5,000 and beyond $50,000. Any involvement in a judicial process will raise such costs exponentially. For our conservative model illustration, we shall assume a “licensing correction” cost of $20,000 after product development is launched.

There are extensive studies on the cost of fixing software defects during development at the Quality Assurance (QA) stage or in the field. Regarding licensing and copyright violations, there is insufficient statistical data to define precisely the cost of addressing IP policy violations. In this case the situation changes with the nature of the violation and the remedies applied.

A policy violation, detected at the QA testing stage, usually involves the testing personnel, the development managers and the actual developers in order to decide what to be done and implement the necessary correction (for example, replace the offending code). This may take more than one person’s day of work and usually ranges between $500 and $3,000. For this example, we will assume $1,500 cost of fixing a problem at the QA stage.

The cost of fixing the problem right at the developer workstation, in real-time as developer brings an offending code segment into his project, is substantially lower. This may take only minutes of the developer’s time and does not involve any other expensive resources. Therefore, the cost, based on the time taken, for fixing issues right at the developer workstation could be between $25 and $60. For illustration in our RoI model, we assume a cost of $40.   In some cases the developer, once notified of IP policy violation at his workstation, can provide an explanation such as “this code is brought in for testing and will be replaced”. The explanation is captured by the tool and kept for the records.

Project and Organisation Size impact on RoI

We have applied the RoI cost model to four different software projects and covering a wide range of organisations in the industry:

–        Large projects with over 100,000 software components (files) and more than 100 developers.

–        Mid-size projects with over 30,000 software components and more than 40 developers in the team.

–        Small projects with less than 10,000 software components and less than 20 developers in the team.

–        Projects with less than 3,000 software components and fewer than seven developers.

For simplicity, we shall assume three distinct approaches:

1. No software IP management action prior to market.
2. Licensing compliance assessment and correction at QA stage.
3. Preventive automatic IP management with final licensing compliance assurance at the build stage.

Results

In summary,

• The larger the project (number of files involved) the higher the amount of external components (open source or otherwise) and the more number of infractions, and there is a higher probability of being “caught” in the field, with the associated cost of “fixing” it.
• Corrective licensing assessment and management can catch the licensing infractions at QA stage, with a resulting cost/correction.

• Preventive licensing assessment and correction, at the developer’s workstation, catches nearly all infractions and cost of fixing each violation in real-time will be lower.

Plugging the numbers into our RoI model reveals the following conclusions on the cost and savings associated with the three IP management approaches described above.

In our cost model we can vary the assumptions and figures without affecting the generality of results.  Ignoring licensing compliance can be costly, and it is difficult to put an upper bound on the cost of shipping tainted software.  Corrective analysis, using automated tools in regular intervals and at QA time reduces the cost exposure significantly.

Combining real-time IP management right at the developer’s desk with scanning at QA or build time reduces cost of potential non-compliance significantly.

Software is ubiquitous these days, in devices and equipment, in desktop applications, and in servers.   Software results from internal developments, are comes from suppliers of sub-systems and chips, outsourced development contractors, open source repositoriesor simply from the previous work of the developers themselves. Software, unlike hardware, is easily replicable, accessed, copied and re-used. Any product that contains software can be potentially infringing on the Intellectual Property (IP) rights of someone or some organisation that originally developed all or part of that software.  Automated applications for scanning software to determine its composition, the pedigree of its components, and the status of compliance to licensing and copyright obligations are key elements of an effective software IP management.

Consciously implementing measures for legal compliance in a software development quality process and incorporating aspects of effective software IP management into the organisation are crucial for any entity concerned with software development and delivery.  Proper licensing and copyright compliance, implemented as part of the normal QA process, can bring savings of up to 40 per cent – 65 per cent, compared to the potential costs of non-compliance. Even better, combining proper QA testing with preventive tools for software IP management right at the developer’s workstation can raise the level of savings to over 85 per cent.

About the author:

Mahshad Koohgoli is the CEO of Protecode, based in Ottawa, Ontario, Canada. He has more than 25 years of experience in the telecommunications industry and specializes in technology start-up businesses. Mahshad has a BSc and a PhD from the University of Sussex.