Christopher Byrne

I reckon this is still one of the best data storytellers that ever lived in the Big Data era. So thought provoking, real and positive. "The idea is to go from numbers to information to understanding" Hans Rosling, I'll just add "To Action" What is your perception of the world population, growth, health and environment? #DataVisualizaton #StoryTelling #Facts #Data

6

Picking randomly from my last post on favourable data-to-value conditions, here’s an expansion (a la GPT) on principle #6: ‘Applications that Externalize Data as a Product and Internalize the Dollar Cost of Associated Data Work’. ******** This principle emphasizes making data easily accessible for the organization while absorbing the costs and efforts associated with data management internally. Here’s an explanation: ### Externalizing Data as a Product - **Data as a Product**: Ensuring data is well-organized, documented, and accessible via APIs, data streams, or other mechanisms. - **Usability and Accessibility**: Standardizing formats and documentation to make data consumable and useful, enhancing data-driven decision-making. - **Interoperability**: Promoting integration across systems within the organization, allowing departments to use data effectively for analytics and decision-making. ### Internalizing the Dollar Cost of Associated Data Work - **Cost Management**: Application run and change TCO cover costs related to data preparation, management, storage, and processing internally, including data cleaning, transformation, storage infrastructure, and quality assurance. - **Operational Efficiency**: Optimizing data-related tasks within the application’s management envelope to reduce redundancy and avoid unexpected costs elsewhere in the organization. - **Data Governance**: Controlling the data lifecycle within the application’s accountability & responsibility envelope, to ensure fitness for purposes and consistent availability while optimizing resources and expenses. ### Contrast with the Opposite Approach - **Externalizing Costs, Internalizing Data**: This alternative involves data remaining siloed within applications, requiring other parts of the organization to intermediate, bearing the costs and effort of extraction, processing, and management. This can lead to inconsistencies, inefficiencies, and higher overall costs to the organisation. ### Example - **Externalizing Data as a Product**: A CRM system makes fit-for-purpose customer interaction data accessible via standardized APIs for marketing, sales, and support teams. - **Internalizing the Dollar Cost of Data Work**: The CRM system handles all data processing and management tasks within its budget. ### Summary This principle advocates for making data accessible across the organization while managing the costs and efforts of data preparation within the application. This approach ensures standardized, high-quality data, optimizing resource use and cost management, and leading to improved efficiency in data utilization across the firm. ******** See the rest of the principles here: https://lnkd.in/ejwn4-eU #datagovernance #techgovernance #systems

5

Been thinking about something lately that's been bugging me in the AI world... We're all excited about these fancy Large Language Models (LLMs), right? But I've noticed we might be getting a bit carried away - it's kind of like buying a Ferrari just to grab a cup of tea or coffee at the café around the corner 😅 Let me share a real example that got me thinking. Last week, while working on a personal project and blog content, was trying to extract company names and dates from some documents. My first instinct? "Let's throw LLM at it!" Then I stopped and asked myself - wait, do I really need this heavyweight for such a straightforward task? Here's the thing: these LLMs are incredible, but they're also like bringing a sledgehammer to crack a nut. Sometimes good old tools like spaCy (about 600MB) can do the job just as well, if not better. Quick story: I switched from an LLM to spaCy for a recent named entity extraction project. Not only did it run smoothly on my laptop's CPU (no fancy GPU needed), but it was also faster and way more cost-effective. Another tool worth checking out is GLiNER - it's even lighter and pretty impressive for custom entity extraction. Look, I'm not saying LLMs aren't amazing - they absolutely are! They're game-changers when you need deep context or handle ambiguous stuff. But maybe we need to stop treating them like the Swiss Army knife of AI? Before deploying an LLM for your next project, ask yourself: - Do you really need all that contextual power? - Could a simpler tool do the job just as well? - Is the extra resource usage worth it? Sometimes the smartest solution isn't the most complex one. It's like that saying: just because you can, doesn't mean you should. What's been your experience? Have you caught yourself overcomplicating solutions just because you could? Would love to hear your thoughts! #largelanguagemodels #machinelearning

37

11 Comments

One of the really hard things in data governance is addressing cross-functional behaviours that have a big impact on data-to-value journeys at scale. Companies start small, with everyone on the same page. Then they scale and elaborate in complexity. Business, tech and data functions diverge, in order to specialise in dealing with emergent challenges. Lines of communication & interaction stretch out, and alignment is increasingly governed by principle rather than practice. Consider the following as a set of alignment principles that create favourable conditions for data-to-value journeys. These journeys go smoother where behaviours are actively nudged towards: [BUSINESS PROCESS GOVERNANCE] 1. Running business analytics/decision/action process workflow on production-grade software rather than meatware. 2. Proactive process design, rather than reactive process mitigation. 3. Persistent machine-readable business process metadata, rather than transient & ambiguous institutional knowledge. [TECHNOLOGY GOVERNANCE]: 4. Capabilities-as-a-Service, rather than as a product. 5. Operating platform utilities and standard patterns, rather than self-service tools and bespoke solutions. 6. Apps that externalise data as a product and internalise the dollar cost of associated data work, rather than the other way around. [DATA GOVERNANCE]: 7. Signal demand oriented, rather than data supply oriented data work. 8. Trusted data sharing networks, rather than monolithic data architectures. 9. Objectively accurate metadata and subjectively fit-for-purpose data, rather than the other way around. …And last but not least - applies to business, tech and data governance: 10. Transparency rather than assurance 😇 #datagovernance #systems

33

5 Comments

There is one big thing you probably don't realise about the diagram (which is excellent btw) from Dylan Anderson below. And that is that Timeliness and Completeness, probably two of the BIGGEST issues in Data Quality are in some ways not data quality issues at all. And that is because if there *is* no data (you could argue) there IS no data quality issue. Why is data not timely? Why does it matter? It happens because processes fail or data fails to get delivered on time. It matters because STAKEHOLDERS FIND OUT before you do --> and why is that? Because your architecture, and probably whatever is orchestrating things, isn't set-up well. The same is true of completeness. Apart from no-one realises. And then the system recovers. And then a stakeholders comes to you and says "hey where is last week's data by the way?" I think this really highlights the importance of ensuring you have robust architecture while also monitoring quality metrics both in a "blocking" sense (i.e. pipeline stops if something fails e.g. in dbt) vs. non-blocking (e.g. Data Quality testing tools or pipelines that are simply data quality tests). You can read some thoughts on this below And give Dylan a follow 😎 #dataengineering #dataquality #dbt

42

4 Comments

𝗪𝗵𝘆 𝗼𝗿𝗴𝗮𝗻𝗶𝘀𝗮𝘁𝗶𝗼𝗻'𝘀 𝘀𝗵𝗼𝘂𝗹𝗱𝗻'𝘁 𝗯𝗲 '𝗱𝗮𝘁𝗮-𝗱𝗿𝗶𝘃𝗲𝗻'... I've had a lot of feedback relating to my earlier post on why organisations shouldn't be data-driven. My suggestion is instead to be 'data-informed', where data is a tool, not a dictator. I'm a strong advocate for using data - but not being enslaved by it, and instead treating data as you would other contributing information - as an important piece of the puzzle that requires consideration. #Data

109

16 Comments

It still baffles me when I see the stats relating to the failure rate of AI projects. One recent study cited 89% of businesses failing to get their AI projects into production! We are making it way harder than it needs to be. Too many people treat AI like some shiny, mysterious tech that requires a “special” approach. The reality: implementing AI is no different than any other technology - it’s just a tool, not some mythical unicorn. The principles are the same: buy-in is crucial, success criteria are essential, and the project must solve a real problem. The biggest misstep? Launching AI as a solution in search of a problem. Without a clear issue to tackle, AI is just an expensive science experiment. Start by defining what problem you’re trying to solve - otherwise, you’re setting up for frustration, not impact. And if you’re not getting buy-in from the team, good luck scaling anything. AI only succeeds when people believe in its potential to make their work better. Setting clear success criteria also keeps things grounded. It’s hard to declare victory if you don’t know what success looks like. AI doesn’t need to be some mystical quest. Get the basics right, solve real problems, and watch that 89% statistic start to shift. #AI

45

6 Comments

I think this is also due to data teams not being able to say no. When people submit Jira tickets, it's not that often that a data team asks firstly whether it's even worth doing the work. Is it aligned with OKRs/goals, is there value in what they want to do with the output... If the answer is no then the data team shouldn't work on it even if they have the time to be honest. The CFO may have seen that the data analysts worked on too much stuff that didn't fit the bill, and felt confident that they could get the analysts working on valuable stuff alongside FP&A, Fin Control etc. Good move in my opinion. HoD is now living their best life, not having to decide what to work on! Analysts work is governed by CFO, this work drives the AE work.

13

2 Comments

The path to a single source of truth isn’t just technical; it’s ethical. By cultivating virtue in data governance, we don’t just manage information better - we become better stewards of the truth itself. As organisations increasingly rely on data to drive/inform their decisions, the real challenge isn’t collecting more information - it’s fostering the right principles. Organisations often focus on technology and tools to manage their data, but at the heart of every successful data governance initiative lies something far less tangible: virtue. Data governance is not just a technical challenge; it’s a moral one. It asks us to balance competing priorities, overcome silos, and create a culture of trust - none of which can be achieved without virtue-driven leadership. Aristotle taught that virtues are habits of character cultivated through deliberate practice. In the realm of data, these virtues might include: - prudence: ensuring decisions are well-considered - justice: balancing the needs of different stakeholders - accountability: taking ownership of data integrity To build a foundation of truth, organisations must encourage collaboration across departments, prioritise clarity over convenience, and reward behaviours that uphold shared standards. Tools and policies alone cannot create alignment; it takes people embodying these virtues to transform fragmented data into unified insight. #Data #AristotleKnows

17

4 Comments

Generative AI has captured the imagination of many, leading to the belief that it can solve virtually any problem. While it’s a powerful tool, it’s essential to understand its limitations to use it effectively. By recognising where Generative AI works best—and where it doesn’t—we can set realistic expectations and maximise its impact. It’s a game-changing tool, but only when applied in the right situations. #GenerativeAI #AI #TechReality #Innovation #BusinessStrategy

59

1 Comment

Scale manipulation is a data visualisation bias that affects the x axis just as much as the y axis. Here is an example of UK interest rates data, where the dataset only captures entries for the dates where the interest rate has changed. If the dataset was to be visualised verbatim, as in the example below, the x axis does not have a consistent and equal time scale. I have highlighted a 7 year period whereby interest rates remained at 0.5%, however the 7 year period is only represented by a single data point. This introduces significant bias, and misses the insight that interest rates remained consistently and unprecedently low for a long period of time. If the x axis on your data visual is date/time based, make sure that the scaling of your time period is equal. This makes the visual easier to interpret cognitively, and also paints a fair and meaningful picture of the trend, avoiding wrong insights and decisions being made. #datavisualisation #trend #interestrates #cognitive #bias #measurement #analytics

3

There's a saying that I love from Sally Bigwood - "graphs are like jokes: if you have to explain them they have failed". Below is a great example of where the graph shows one thing - but with a more detailed explanation is found to display the complete opposite. #datavisualisationfail

20

1 Comment

Tedious Christmas 🎅 Post Incoming A non flavoured chocolate orange seems to be out of whack with the values and expectations of a Terrys Chocolate Orange. It’s the equivalent of having spent years on your analytics platform, a beautiful warehouse and analytical layer and then sticking 10 raw tables and 8 csvs into a powerbi model and putting it on import mode to hide any performance issues because you couldn’t be bothered to follow the pattern of your data warehouse/analytics platform anymore. This is not OK. FYI I’ve deducted 10% from my mothers Christmas budget next year as punishment for bringing this into my life. #dataleaders #dataleadership #datawarehouse #dataengineer

49

15 Comments

The Databricks Data & AI world tour in London was brilliant! I still need to synthesise my thoughts and notes, but a key observation is that data is still king. From retail, to aviation, to energy, the big challenges across industry are, largely, centred around data quality, data governance, and siloed data. #Databricks #Data #AI

42

4 Comments

tell me you're a data professional without telling me you're a data professional... when I build brio train tacks, I build them only using track pieces. Once it's up and running then I will swap out pieces of track for the fancy bits i.e a windmill. Foundations over fancy features for release 1.0. #dataleaders #dataprofessionals #dataleadership #cdo #dataanalytics

54

25 Comments

Understanding, knowing and trusting data valuations when calculating the value of AI data is a critical aspect to pricing the AI asset both for external use and even internally by an organisation. Data Valuation Partners is a leader in this space.

1

It's a big birthday - Data Orchard's Data Maturity Assessment is 5 years old today! Nearly 12000 users from over 1000 organisations. Here's a celebration blog to mark the occasion #data #analysis #datamaturity #data4good #CEOs #nonprofits #charities #socents

11

5 Comments

Apparently the bulk of Tableau sites aren't set up well. Yes, in my recent survey, 94% of respondents thought their Tableau sites could be better. And half of those thought their Tableau site has plenty of room for improvement. I'm surprised by these stats. I've carried out several Tableau site reviews, and, in the main, most are quite well run. Yes, there's often some room for improvement, but overall they're in a fairly good state. Smart use of projects, groups & data sources generally lead to a good Tableau site – at least from the end user perspective. What are the main issues causing people to say their site isn't well set up? #tableau #tableauserver (link to the survey in the comments)

4

3 Comments