Danny Sandwell, Director of Product Marketing at Erwin by Quest
Currently, most organisations are not using the data available at their disposal to achieve their strategic objectives. While IDC predicts that the amount of data globally will reach 175 zettabytes by 2025, many enterprises are still behind and not wringing the true value from the data they hold. Everyone wants to be part of a ‘data-driven organisation’ but the process can be drawn out and labour intensive without the right management of data.
One of the most key factors to control in your journey to ‘data empowerment’ is the management of what is called metadata.
Quite simply, metadata is data about data. It’s generated every time data is captured at a source, accessed by users, moved through an organisation, integrated, or augmented with other data from other sources, profiled, cleansed, and analysed. Metadata is valuable because it provides information about the attributes of data elements that can be used to guide strategic and operational decision-making.
Metadata management is the hero in unleashing the real potential of enterprise data. When done strategically, metadata management helps you understand the data you have, where it comes from and how it flows through the enterprise. It sheds light on specific characteristics about the data, such as its accuracy or sensitivity (for example personally identifiable information PII), as well as the rules and restrictions that govern its use.
Crucially, this metadata allows you to understand what data means within the context of business. At its best, metadata management fuels enterprise data visibility, automation, data governance and collaboration across your enterprise. Effective management involves creating and sustaining an enterprise-wide view of and easy access to underlying metadata.
The challenge:
But for many organisations, getting a handle on metadata management can be an uphill struggle.
Often, they find that their data infrastructures have been cobbled together over time with disparate technologies, poor documentation, and little thought for downstream integration. Likewise, the policies, processes and tools that define and control data access and roles can be severely lacking. The business context is often limited or missing, and common terms haven’t been standardised and adopted across the organisation.
Finding, ingesting, integrating, linking, sharing, and analysing metadata usually depends on manual processes and takes a lot of time, money, and specialised technical resources. Very few enterprises have enough trained personnel, money, or time to do this effectively using their traditional manual approaches.
Trying to keep pace with modern enterprise data requirements whilst relying on these messy foundations is a recipe for disaster. The applications and initiatives that depend on a solid data infrastructure may be compromised, rendering faulty analyses and insights. Without being able to take full advantage of analytics tools, an organisation can’t become data-driven — or worse, it could end up going in the wrong direction.
Automation is the answer:
As with numerous other IT and data management processes, automation has arrived and completely changed the game, making metadata management much more practical and the benefits much more achievable.
By automating the process, organisations can achieve the gold standard: building, enriching, and maintaining a central metadata repository that documents all data sources, data movement processes and data consumption throughout an organisation. With metadata visible, controlled, and accessible from a single reference point, its utility to the business increases by orders of magnitude. The automation tools are utilised to identify and harvest data sources, processes, and consumption for technical metadata. Teams can then curate these collected data assets and associate them with additional technical, business and quality characteristics for enriched documentation, organisational clarity, and data understanding.
The benefits:
As the amount of data continues to grow, the benefits brought by well-structured metadata management are getting more recognition from enterprise teams due to the proliferation of data analytics initiatives.
The first advantage resides in better data quality, as data issues and inconsistencies within integrated data sources and targets are identified in real time, resulting in higher efficiency. This also helps knowledge workers by freeing their time from finding, understanding, and resolving errors.
Secondly, as the regulations such as GDPR, HIPAA, PII, BCBS and CCPA have data privacy and security mandates, requiring sensitive data tagging, lineage documentation and data flow traceability, metadata management helps ensure regulatory compliance. Knowing what data exists and its value potential enhances digital operations, drives digital innovation, and improves digital experiences.
Lastly, the reliance on automated, repeatable metadata management processes results in greater productivity. With the business driving alignment between data governance and strategic enterprise goals, leaving IT to handle the technical mechanics of data management, organisational objectives can be more effectively.
However, it is worth mentioning that to achieve best results, the importance of data accessibility must be understood, as to create a trusted picture, metadata management tools must be able to identify all the data, including data stored in databases, data warehouses, data lakes as well as data currently in motion. For an enterprise to be able to rely on automated and repeatable metadata management processes that could result in greater productivity, all stakeholders must collaboratively participate to drive the alignment between data governance and strategic goals.
In the past metadata has always been a secondary asset, however, with the current emphasis on its efficiency and drive for value capitalisation, its role will continue to grow. Therefore, as we step in 2022, business leaders are advised to focus on what you can actively do with metadata rather than simply how much you can store, as well as leverage and activate it to achieve faster and greater data insights.
About the Author
Danny Sandwell is an IT industry veteran who has been helping organizations create value from their data for more than 30 years. He is responsible for evangelizing the business value and technical capabilities of the company’s enterprise modeling and data intelligence solutions. During Danny’s 20+ years with the Erwin brand, he also has worked in pre-sales consulting, product management, business development, and business strategy roles – all giving him opportunities to engage with customers across various industries as they plan, develop and manage their data architectures. His goal is to help enterprises unlock their potential while mitigating data-related risks.
