Dealing With The Metadata Mess

Courtesy of Optimize

Start thinking about where metadata exists in your company, and before long you'll be breaking out the ibuprofen. Content management applications. Business intelligence repositories. Service-oriented architectures. Data-quality management systems. Although vendors haven't given us a vehicle for snap-your-fingers consolidation, Forrester Research analyst Rob Karel has some suggestions about how you can start thinking about the problem and derive a solution.

Q: Let's start with your definition of metadata and why it's such an issue.

A: My definition of metadata spans multiple technology platforms, and the challenge is to drive the necessary architectural convergence of these different platforms to solve more-complex business problems. And although different applications manage metadata, they don't manage the different flavors of metadata that build the foundation for structured data management, such as those in ETL [extraction , transformation, and load] applications, data warehouses, business intelligence, or SOA technologies.

Q: What should CIOs be thinking about metadata?

A: They should be thinking about metadata only in terms of the business problems they're trying to solve, rather than focusing on the technology they're married to. Metadata needs to rely on those [aforementioned] technologies working together in an integrated fashion.

Q: Is it possible for applications to share their metadata?

A: Not all metadata needs to live to a physical repository. That's technically unfeasible and probably even unnecessary. You don't need it all in the same place, because it serves different purposes. You have to build an architecture with technology that can manage the appropriate capabilities, whether it's Web services with a feed to multiple repositories or business intelligence.

Remember, there's going to be business intelligence metadata that's going to be relevant only to someone doing BI, and that's OK. But you have to think about how this metadata is relevant beyond this silo. What are you trying to do, what metadata do we need to do it, and how do we find it?

Q: What vendors are we talking about?

A: There isn't a single technology, although we may have stuck some technologies together with duct tape. Most of the stand-alone metadata repositories specialize in structured data. There is no easy answer. The standards efforts are going for low-hanging fruit, with specific business processes coming later. You may have to come up with your own definitions—start with your business model and work down.

Q: Assuming that you can do this, what value can you derive from integrating metadata?

A: It's a communication layer for cross-functional projects. It enables that next level of capability, because you're not stuck in silos within the technology. You can get a 360-degree view of a customer. You can see employee data more clearly. It gives you the ability to take information or knowledge from disparate systems and allow IT professionals to see the value in a cross-functional effort to enable a simple capability. The overall goal is to share knowledge across the enterprise, as opposed to having it retained by just a few individuals, and make sure it's not duplicated.

Q: What should IT do now?

A: IT should start working from the bottom up and from the top down. Vendors within these silos have made vast improvements in how they're allowing end users to capture data within their own environment. Make sure you're doing the best you can to make sure the metadata you've captured is as high-quality as possible.

Q: Who's doing this well among the vendors?

A: There are data integration vendors that are making strides in expanding their own ecosystem; for instance, IBM is working on master data management issues and is thinking about unstructured data as well. That said, even though the vendors are thinking about it, they haven't gotten there yet. Very few vendors even have offerings that cross all application segments. Some of the bigger companies do, because they play in several market spaces, and they see the value for their own offerings.

Q: Any other challenges?

A: Metadata is the least popular of the initiatives without a visible ROI. Now data quality is getting its day in the sun, but five years ago, data-quality measures couldn't prove their own ROI. Metadata is where data quality was five years ago.

Q&A conducted by contributing Web editor Howard Baldwin.