Matia

🔥 Netsuite Connector now live 🔥

📣 Attention TLV Data Community 📣 Join us at the next Snowflake User Group - Tel Aviv on March 4th. Multi-tenancy and data ingestion/ETL at scale are critical for modern data teams. Our CTO, Geva Segal, will be diving deep into strategies for optimizing ingestion, managing multi-tenant architectures, and reducing operational complexity, all powered by Snowflake. ❄️ Come ready with questions and let’s talk about making data work smarter, not harder. 🔗 Link to signup: https://lnkd.in/gC2ZVC7Q #SnowflakeTLV #DataEngineering #DataOps

You know you're a data engineer when... 🧠 ✅An analyst pings you to get "just a few tables" to the warehouse and you know it's going to take all day 🙄 ✅You're tired of explaining to people that data warehouses don't hold every piece of data your company owns ✅ You see SELECT * in production and physically cringe ✅ You've been personally victimized by a schema change ✅ No one (even engineers) really understands what you do What would you add?

The results are in. Matia's most utilized connectors in January: 1 - Snowflake 2 - #PostgreSQL 3 - Salesforce - special shoutout to all of our customers using the newly improved Salesforce Formulas 4 - Google Sheets 5 - HubSpot Get the full list of Matia connectors here: https://hubs.la/Q036hnTT0

What do Thor, schema management, and CDC have in common? Turns out… not a whole lot. That is, until you're tasked with building data pipelines for a very realistic Marvel strongman competition. If Thor’s power level suddenly changed from an integer to a float, your pipeline would feel the impact. Schema drift and error cascades can wreak havoc on your data, breaking transformations and dashboards before you even know there’s a problem. That’s why resilient pipelines need proactive schema management, Change Data Capture (CDC), and built-in error handling. Learn how to build data pipelines that can handle whatever comes their way—without grinding everything to a halt. Because even the God of Thunder deserves clean data. ⚡️🔨 Link in comments.

New office. New milestone. ⬇️

Data teams deserve a break. Seriously—how many different tools are you switching between just to move, monitor, and manage data? 📥 One (or more) for ingestion 📤 Another to activate data 👀 Something else for observability 📚 And, of course, how will my business users know what data we have - they want a catalog. Feels like half the job is just keeping the tools talking to each other. At some point, you have to ask: Is this really the best way? And that got us thinking 🧠 https://lnkd.in/g7TA4vBh

And we're live. Catch the first edition of Sync or Swim 🌊 - Matia's newsletter about data and anything else in the data world we find weird/interesting at the moment...AI and space exploration, anyone? We enjoyed writing it. Hope you enjoy reading it. If you still like us after reading it...like/comment/subscribe/dance

Every data engineer knows that scaling ingestion volumes isn’t for the weak. 💪 From unexpected network issues to rigid architectures and manual monitoring headaches, there are plenty of pitfalls that will trip you up. Scroll the list below for some of the main ones we see from our customers. ⬇️ And of course, link in comments for more on how to remedy and avoid them.

There's a lot of chatter about deeper data observability, but what exactly does that mean? 🤔 There are many ways we can deepen observability and take it from good to ➡️ great at different points along the data journey. At ETL/Ingestion 🟡 Good: Detecting if an integration failed to run 🟢 Great: Knowing which tables sync, their sync times, and clearly seeing if a sync is taking too long and why At Transformation 🟡 Good: Detecting abnormal job runtimes for recurring jobs 🟢 Great: Monitoring dbt via in one platform that consolidates all your dbt tests At Reverse ETL/Activation 🟡 Good: Getting an alert that a sync has failed 🟢 Great: Being able to pause an upcoming sync (and keep your destination clean) when an upstream critical monitor is failing Anything else you'd add ?👀