Black Tiger Insights
2
min read

How do I organize my data?

Black tiger

Get more value out of you data and freedom to use it effectively

With 20 years of expertise in data technology, Black Tiger is regularly confronted with the difficulties faced by businesses in their battle with their own data.

This battle could be summed up as ‘How can I put my data in order?’ to get more value out of it and more freedom in how it is used.

We can reasonably divide the difficulties faced by companies into 4 categories:

  • Data accessibility
  • Traceability
  • Data quality
  • Making the most of it to work efficiently

 

1. Data accessibility is the first challenge for businesses, due to the complexity of their existing systems:

  • Data spread across a large number of heterogeneous systems, both in terms of infrastructure (public cloud, private cloud, on premise) and technologies (old legacy technologies vs. new applications, data lake, BI, etc.), backed up by more ‘modern’ technologies, document data bases, graphs, index engines, managed services, ....
  • Data siloed by application, and therefore duplicated
  • Different types of data, from structured to unstructured
  • Problems identifying the ‘master data’, its owner and its experts
  • The identification, and even more so the use, of data that can be ‘joined’ between heterogeneous systems is a very important issue.

 

2. Data traceability is a second difficulty since, in its absence, and apart from the lack of compliance that this generates, it does not allow the company to be in control of its data. Lack of control prevents the company from understanding inconsistencies and errors and, as a result, slows down the correction process considerably.

  • Once it has been accepted that transformations, of varying degrees, are necessary inorder to be able to use its data effectively, it becomes essential to keep very precise records of these transformations. Not only (in some cases) for regulatory reasons (e.g. GDPR), but also for the very operational needs of error recovery, rollback, changes to models and rules, etc.
  • Trace ability is only fully achieved when the transformations carried out during ingestion are effectively usable, in particular by enabling searches, automatic processing, etc.
  • Tracing must also be possible beyond ingestion, once the data has been made available to other applications.

 

3. Data quality is a third difficulty which seems to be obvious to everyone and a discovery for no-one. It remains a definite problem that companies struggle to solve.

  • Data quality is always a challenge that can quickly become overwhelming for non-specialist companies...
  • It's always a delicate balance between totally generic issues and totally specific particularities.
  • The challenge for companies is therefore to find the right balance between what is a completely standard element, where it is essential not to go out of our way to ‘reinvent the wheel’, and what is truly specific to my sector, to my company, where we need to focus our energy so as not to get bogged down in an overly generic approach that does not correspond either to my operational reality or to my objectives.
  • This balancing act is complicated when you're not a specialist in the field, and, when you don't have the technology to easily marry these two seemingly contradictory approaches!
  • Another challenge facing companies, which is always at least underestimated and at worst completely ignored, is the gap that exists between the reality of data and its ‘idealized’ representation by the business...
  • Finally, another challenge, which stems in part from the previous one, is to be able to treat the mass and the exceptions differently. This is something that is even more difficult for non-specialists to grasp because it depends on the use case for which the data is intended!
  • An effective solution must automatically process as much data as possible, reserving the correction of exceptions for business operators. In such cases, it must also be possible, where possible, to re use the correction made to automatically process future exceptions of the same nature.

 

4. The final difficulty is the effective use of data. At Black Tiger, this can only be solved by providing companies with a 360° view of suppliers, contracts, partners, products, customers, employees, etc. Such an approach can only be achieved thanks to a technology that guarantees the 7 most challenging and fundamental data treatments: Accessibility, integration, quality, de-duplication, aggregation, traceability and compliance.

  • The key concept when it comes to exploiting data is adapting processing to usage. In the world of big data, there is no such thing as universally correct/ deduplicated/ aggregated/ compliant data, but as many data sets as there are use cases! An approximation, a rounding-off, etc. can be perfectly legitimate for a given use case, but catastrophic for another. Likewise, having consent for an email may be necessary for a given purpose, but not at all for another.

 

Businesses are confronted by a market of heterogeneous solutions, blurring the path to follow and complicating the project process.

The sheer number of offerings on the market drowns customers in a sea of highly fragmented technological possibilities, making it difficult for them to come up with an overall, coherent analysis of their problems.

Moreover, technological diversity forces customers to try and solve their problems by adopting several technologies, making the project difficult, costly and sometimes never-ending.

In the end, Black Tiger very often finds itself having to structure all of this in order to guarantee effective data deliverability.

Written by Michal KOLATAJ

Weekly newsletter
No spam. Just the latest releases and tips, interesting articles, and exclusive interviews in your inbox every week.
Read about our privacy policy.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.