Yiftah Porat

Very much in agreement with this. A knowledge graph can be very useful, but it can also be a bottleneck if not designed correctly.

9

4 Comments

➰ APIContext's latest insights on cloud-based API performance have been featured on Nordic APIs! In the 2024 report, we delve into crucial aspects affecting API efficiency: 🔐 Authentication is key. Ensuring robust authentication mechanisms in APIs is crucial, yet offloading this function can introduce complex challenges. 📖 DNS configurations matter. How you handle DNS settings can significantly impact API performance, particularly when tied to authentication processes. 🌐 The role of third parties. Critical third-party services play a significant role in 🔗 API transactions, underscoring the need for careful integration and monitoring. ⛅ Not all clouds are created equal. Differences in cloud services can drastically affect API performance, an essential consideration as APIs account for over 80% of all internet traffic. #APIs #CloudComputing #TechInsights #APIContext

15

1 Comment

Apache Kafka is dead, just 100 years after Franz Kafka! Kafka's books lived on. Long Live the Kafka API! The existing Apache Kafka had to die. It’s a good thing, for several reasons. 1. Kafka isn’t cloud-native. Managing brokers and queues has been a headache for most messaging systems, and Kafka has been one of the best examples. Part of the problem has been tightly coupled storage and compute. Kora is Confluent’s replacement, but they’re not alone. RedPanda and Warpstream are others. 2. The innovation is happening around, not within Kafka. There’s a great ecosystem around the Kafka API, and it’s driving new innovation. The next generations of real-time data integration, streaming analytics, and all kinds of stream-based apps are all evolving. Beyond working on modernizing Kafka, the Apache Kafka project needs to invest in supporting all this innovation. 3. Kafka doesn’t understand data, or data integration. Kafka has always focused on messaging and stream processing, not data management or integration. New technologies have always gotten built on top of messaging to integrate processes and data, or process events. This has happened with TIBCO Rendezvous, JMS, Kafka, even (async) APIs. In the case of real-time data integration and data sharing, something needs to manage data across sources and destinations. You need to manage and integrate data schema from sources to destinations with transformations and workflows. It needs to be visual, flexible, and fast to change. That’s not Kafka. Kafka has queues and a schema registry. You code. But the Kafka API is allowing the next generation of tools to enter the market faster because you can access so many tools, and so much data, via support for the Kafka API. So what should happen to Kafka and around it over the next few years? 1. Kafka will become cloud-native. Either Confluent contributes a cloud-native version, one or more Kafka API compatible technologies take off and create multiple market segments, or both. It’s probably both. 2. New messaging protocols will emerge. Kafka is for data streaming. We need more. There’s NATS and a host of other messaging technologies for IoT. There’s Gazette for exactly-once delivery with built-in stream-store-replay that supports data integration, analytics, and data sharing better. We still need more for transactional messaging. And something will always exist outside the firewall. Most of these technologies use the Kafka API in some form. 3. Data schema and workflows will hide messaging namespaces. I say this with the utmost respect: message namespaces have no business value. If managed directly they will slow down change. They must be driven by higher-level tooling. This is how Estuary Flow works. You can connect to Flow and receive data using the Kafka API. Is Kafka dead? Going cloud-native? What do you think will happen to Kafka? Apache Kafka #ApacheKafka Confluent Redpanda Data WarpStream Estuary

714

141 Comments

Whether you're handling millions of requests per second, managing live chats, or running a high-speed leaderboard, Redis operates entirely in memory, making it incredibly fast. 🚀 Redis excels in real-time performance and in-memory operations for blazing-fast caching, messaging, and more. But Redis is just the beginning! 🌊 In my latest video, I explore 6 other lesser-known databases that can transform your projects, from handling complex relationships to AI-driven vector searches. Don't limit your tech stack—discover these powerful tools. Check out the video here: https://lnkd.in/dE7_dP_A #Databases #UncommonDatabases #Redis #ArjanCodes

3

How does Debezium react to schema-related modifications behind the scenes? The short answer is it depends. 🙃 Debezium cannot expose schema changes across the databases it supports in an agnostic way. There are currently two fundamentally different approaches: 1️⃣ Explicit schema change events as a result of applying DDL statements can be published to a dedicated channel, which is supported by MySQL, SQL Server, and others 2️⃣ For PostgreSQL, DDL changes aren't currently supported, and as a result any schema modifications are only reflected as part of the data change events themselves 💡 Learn more about schema evolution in change data capture pipelines https://dcdbl.co/3C2XrsN

9

This week, I've gotten interested in testing how well multimodal modals handle document text extraction. The results are rather surprising... surprisingly good. 🤯 I've completed some testing with Anthropic Claude 3.5 Sonnet to see how it compares to cloud services like Azure AI Document Intelligence. You can read the details in a new blog post, which shows the structured JSON output returns from a simple prompt against an document page screenshot. Since these models have now become a reliable alternative, we will add support for them in the Graphlit, by Unstruk Data preparation workflow. https://lnkd.in/gpzFVwFy #unstructureddata #pdfextraction #multimodal #llm

14

🚨 Anthropic Drops Major Updates: New Claude Models & Computer Use API! 🔄 Two New Models Released: 1️⃣ Upgraded Claude 3.5 Sonnet - Available now on Anthropic, Google Cloud Vertex & Amazon Bedrock - Outperforms previous version on most benchmarks - Significant improvement in coding (SWE bench: 33.4% → 49%) - Better at agentic tasks & tool use 2️⃣ Claude 3.5 Haiku (Coming Later This Month) - Initially text-only (vision support coming later) - Surpasses original Claude 3 Opus on some tasks - Faster & more cost-effective option - Exceptional performance for its size 🖥️ New Computer Use API: - Direct computer interaction capabilities - Can perform web searches, edit documents & navigate interfaces - Uses basic commands (mouse movements, clicks, typing) - Similar to autopilot technology - requires supervision - Currently focused on web-based applications ⚠️ Safety Considerations: - Need for careful implementation - Potential for errors (as seen in testing) - Recommendation: Use on separate dedicated machine - API documentation and safety protocols pending 💡 What's Next? - Claude 3.5 Opus still anticipated - Further development of computer interaction capabilities - Expanded vision support for Haiku This release marks a significant step forward in AI capabilities, especially for coding and automation tasks! 🚀 #AI #Anthropic #Claude #ComputerUse #Innovation #AITechnology https://lnkd.in/edUXq6Rv

48

3 Comments

Optimize #PostgreSQL JOIN performance with a comprehensive troubleshooting guide. Learn how suboptimal indexing affects JOIN efficiency and discover tips for writing optimized queries to enhance database performance. #PostgreSQLPerformance @MinervaDB #dba https://zurl.co/M9VL

2

Optimize #PostgreSQL JOIN performance with a comprehensive troubleshooting guide. Learn how suboptimal indexing affects JOIN efficiency and discover tips for writing optimized queries to enhance database performance. #PostgreSQLPerformance @MinervaDB #dba https://zurl.co/M9VL

1

💡 Optimizing Costs with Dragonfly: A Technical Breakdown 💻 For technical teams looking to maximize performance while minimizing cloud infrastructure costs, Dragonfly offers significant advantages. By cutting down memory overhead, leveraging CPU-specific optimizations, and using a shared-nothing architecture, Dragonfly is designed to be resource-efficient while maintaining high throughput. Whether you’re scaling horizontally or aiming to optimize performance, Dragonfly can help you reduce operational costs without sacrificing speed. In this blog, we break down the technical strategies that Dragonfly uses to cut costs and improve efficiency. 🔗 Read more here: https://hubs.la/Q02RDNVr0 #CloudOptimization #TechEfficiency #DevOps #DatabasePerformance #Dragonfly#CostReduction

3

#LangTest 2.3 is out! Major release of the #OpenSource library for #ResponsibleAI testing of custom #LargeLanguage Models. New features include automated Multi-Model / Multi-Dataset Testing, Drug Name Swapping Tests, Prometheus Integration, LLM Safety Testing, and Improved Logging. Release notes: https://lnkd.in/gvW_GCx7 Get Started: https://langtest.org/ #generativeai #ai #ethicalai #aitesting #softwaretesting #llm #llms

23

Business success when it comes to building a data ecosystem to fuel a data informed culture and enable new capabilities is incredibly dependent on your architecture. Credit to Wandelson Bezerra form Banco Itaú. When developing a business strategy that includes solutions that are reliant upon data then we need to step back and look at the horizon. Simply building a data ecosystem and capability to address yesterday's or today's opportunities is likely not going to serve tomorrow's. With the pace many businesses operate it can be very easy to overlook bigger picture thinking and simply make small improvements or apply band aids. This may get us to the next step but over time in a data ecosystem can create volatility, risk and operational/technical debt. What then happens is that over time this multiple to the point where the business has little choice but to address, often a time consuming and costly exercise. The opportunity upfront becomes that of being able to design an architecture than has longevity. Now, whilst a business may not acknowledge of appreciate this in the same way a data leader does it's about balance. How do we maintain business cadence whilst delivering tangible value as the same time we're building future fit foundations. Wandelson shares and article which outlines 6 principles you should consider when looing at the design of your modern data architecture ⚒ View Data as a Shared Asset. ⚒ Provide User Interfaces for Consuming Data. ⚒ Ensure Security and Access Controls. ⚒ Establish a Common Vocabulary. ⚒ Curate the Data. ⚒ Eliminate Data Copies and Movement. If you need an analogy for this one, think about building a house for your family. How would you think about your needs and how they may change over time. #dataarchitecture #datafoundation #datacapability #datastrategy

9

Rice's Theorem : Two Line Change Caused $50 Billion Loss = I often think about effective ways to understand software behavior, especially in the context of distributed systems. One important sociological disjunction we see when we try to improve our understanding is the disjunction between - I put it loosely - management and workers (in particular, incident responders) and what they see as the underlying basis for software failure. Again, this is approximate, but let me characterize two views. In view A, software runs stably unless acted on by an external force, and that force is overwhelmingly human-originated. Therefore, when something goes wrong with the software, it’s overwhelmingly likely to be a human who did something wrong. Restoration of service is a task of reverting the human-originated change, and prevention of error is closely tied with preventing humans doing “bad things”. (There are many possible critiques of this view; I merely relate the core proposition.) In view B, software is capricious and unstable-by-default, the environment is constantly changing, and if there is a failure, it can be the result of the combination of a collection of side-effects from a thousand different actions that are causally very distant from each other. It is therefore not useful to say a human is at fault. (Critiques of this view are also available: for example, the notion that for deterministic software, at some root of some chain must be a human action.) It is important to say that these views are not pure logical opposites, and could both plausibly be claimed to be wrong, but are distinct models about the operation of software that drive a view on how to attribute sources of failure in software (and how to prevent same, restore service, etc). They are roughly analogous to Dekker's Old View versus New View on human error. For completeness, we would stereotypically say that management holds view A and the workers hold view B. = In computability theory, Rice's theorem states that all non-trivial semantic properties of programs are undecidable. A semantic property is one about the program's behavior (for instance, "does the program terminate for all inputs?"), unlike a syntactic property (for instance, "does the program contain an if-then-else statement?"). A non-trivial property is one which is neither true for every program, nor false for every program. The theorem generalizes the undecidability of the halting problem. It has far reaching implications on the feasibility of static analysis of programs. It implies that it is impossible, for example, to implement a tool that checks whether a given program is correct, or even executes without error. We need to help holders of view A understand that it is not true, and that their model that the software is inherently stable is incorrect. To the extent it’s useful, it is therefore possible, and true, to say to leaders “software cannot be shown to be stable, and so it’s safer to assume it isn’t”.

59

33 Comments

The fine folks at Anthropic have released yet another capability to make Claude even more powerful: MCP. MCP stands for "Model Context Protocol", and it gives us one standard for connecting all of our data into the Claude eco-system for use in prompting. In this demo, Alex, from the Anthropic team, shows how to wire Claude up to the web for access to real-time info off the internet. This is something that was not possible up to this point with Claude. Check it out!

5

With the discussions around llms.txt (https://llmstxt.org/) and Model Context protocol or MCP (https://lnkd.in/gDZRGNAa). It's clear we're going to need a new set of interfaces and standards to provide context and capabilities to LLMs and the coming wave of Agents. There's a significant amount or risk and opportunity around discovery and validity of content that is going to have huge ripples as we reconsider how people discover information, products and services. This shift in the interface of value from search/social network to Agent is going to be an exciting space to watch. David Berlind, might be an interesting time to consider an MCP relaunch of programmable web #Agentic #interfaces #API #AI #futurism

2

2 Comments

#Knowledgegraphs significantly enhance #AI strategies by providing a foundational understanding necessary for reasoned judgments, training, and validation. They enable AI systems to incorporate essential knowledge into their operations, thereby improving conversational abilities and statistical insights. This integration of organizational intellectual property into AI strategies enhances market differentiators and competitive edge.

1