Chris Maden

Had a chat with Eric Meyer and Brian Kardell about CSS masonry, does it belong in the grid spec or not? With a bit of a discussion about how and why we ask for developer feedback on CSS. https://lnkd.in/d3C-_dwq #css

33

Anthropic released another paper on their ongoing work towards interpretability of large foundational models like their Claude 3 family, built on previous work by many. Their work continues to be enlightening and inspiring. The findings in this work are generally similar to previous findings on smaller models but scaled up to Claude 3 Sonnet: 💡 they find large numbers of "features" corresponding to combinations of neuron activations and these features map to concepts that we recognize/understand: for example, features for transit infrastructure, particular people, concepts (like bias and "objects lacking sentience"). This might sound familiar from classical machine learning, but the interesting thing is that there is no single thing/neuron encoding each of these concepts, it's a particular combination of neuron activations that embeds these concepts in "superposition" in the network. 💡they observe a sort of "distance" of these feature activations that arrange themselves "near" semantically similar features. When they use their sparse autoencoder approach to learn more and more features they observe that the level of abstraction in the detected features becomes more granular, in ways that are interpretable and meaningful. 💡when they artificially fire these activation patterns during inference, it affects the output in ways that relate to the "feature". For example, asking Claude "what's the most interesting science" normally it responds Physics, but if you clamp the feature activations for the feature "Brain Science" to 10x their max values then Claude responds: Neuroscience. Ways to interpret _and_ influence the model. Great readable work [1] along with an interactive explorer of some of the "features" themselves [2] 🎂 There's an easter egg too: if you scroll in the feature explorer tool and try and select one of the "Randomly selected features from 1M" you get a message "if you want to see the rest of the features, we're hiring!" with a link to their recruiting page :) [1] https://lnkd.in/gZbparwp [2] https://lnkd.in/gC9eus53

19

This is a concept I talked about in my latest conference talk. tl;dr When everyone focuses on solving the user's problem, everyone helps each other and everyone wins.

5

As I noted a long time ago, the book on the left is based on those books on the right (and dozens of blog posts and videos). And now the PDF version of the book on the left is free. (Enjoy) https://lnkd.in/gCNggRXg #scala #functionalprogramming

172

13 Comments

I’ve spent pretty much the last 15~ years of my life working in OSS and I don’t think I’ve ever had a more exciting year than 2024! Everywhere I go, I see the power of OSS opening up opportunities for collaboration and innovation. I think it’s undeniable that this has been a year for truly open source software (and open governance) because of the momentum behind Apache Iceberg and now Apache Polaris (Incubating) [https://lnkd.in/gmBQthbz]  — and it looks like I’m not alone! Apache Iceberg  has been selected as BigDATAwire’s latest ‘Readers’ Choice: Top 3 Big Data and the AI Open Source Projects to Watch’ and ‘Readers’ Choice: Top Data and AI Achievement’ awards. In addition, Apache Polaris was also listed as a top open source project to watch. When I joined Snowflake, I was proud to be a part of a company that was embracing Apache Iceberg and truly OSS by collaborating on the development of Apache Polaris and pushing great new features into Apache Iceberg. Snowflake has been making huge strides in the open data community, and I’m so excited to see our joint efforts receiving such kind feedback from peers. It’s both inspiring and rewarding to be working towards actually providing value, without vendor lock-in, to big data users everywhere! #BigDATAwireRCA24 https://lnkd.in/gZAAAmWT

103

1 Comment

May be my ignorance. But what stops generative AI bots being orchestrated by a bad actor to create ‘community notes’ en masse for deliberate dissemination of misinformation which isn’t fact checked? Counter point. I’ve seen community notes worth well on X (so far!). But the above is a legitimate risk if there are no counter measures in place. However, there is another risk these could be interpreted as censoring it not actioned correctly.  At ‘this’ point there is no right answer. The question is: ’What is the greater risk? Hypothetical bad actors driving misinformation or corporations changing the rules and not having enough safeguards in place potentially due to their bottom line?’ Let’s hope neither. https://lnkd.in/eFvETfMj

4

Reading Ted Mabrey's article on FDEs at Palantir (https://lnkd.in/dEeweq2g) I'm completely in agreement with his point that you've never "finished" building a product, thus your mission is to expand the "perimeter" of your product as fast as possible. IMO one can see this as a bit like the universe: it's always expanding. Not only do you want to keep expanding the use cases your product addresses, but there are ever more use cases popping up. Going further, if we imagine the number of use cases as a linear dimension, and the effort required as the surface area, unsurprisingly adding more use cases to your already complex product takes much more than linear effort... So not only is a product never finished, it becomes ever harder to expand its perimeter. Thus, building an ecosystem of products (from partners, or a well-integrated solution of one company's products) is likely to work better than one "super app.". The former is a collection of small spheres, each of which can expand to targeted incremental use cases with modest effort, versus the latter which is a huge and very hard to grow.

8

Interesting survey of the various applications of LLMs in journalism, from thoughtless to thoughtful: "Faced with the chaotic, messy reality of everyday life, LLMs are useful tools for summarizing text, fetching information, understanding data, and creating structure. But always, always with human oversight: guiding the summaries and then checking the results, teaching the bot to navigate an audit report, deciding when and when not to put the bot in control, and designing the rest of the software that powers the site. In all of these cases, it's humans first and humans last, with a little bit of powerful, generative AI in the middle to make the difference." https://lnkd.in/g5vXs5vz

2

1 Comment

Monitoring usage for Canopy AI has been interesting. Some people frame things super well and get great results. Others ask a four-word question, and somehow still get some okay results (just because we have a lot of content + I categorized everything for the RAG), but not always. We do some prompt augmentation, but we should do more. Currently, we inject things in certain scenarios or when it fails. Ideally, we would have a two-pass system for every request, where everything goes through a prompt improver. You can't assume users will ask the right things. All kind of obvious things I had already considered but left for a later version after initial usage. The cool part is this would've been hard to do last year due to speed/cost; now you can just "pipe" things multiple times using the same high-quality model you call at the end.

5

For fun I made a presidential polling website with real-time electoral results: https://ileantoward.com It requires semi-live participation and votes can be changed at any time. Only visitors from the USA can participate. Congressional district is determined by IP address. The idea is to enable a coordinated voting process, where selections are non-binding and negotiable rather than blindly cast once. Kind of like a presidential caucus, but national and over the Internet. Obviously it uses Fanout for pushing updates to the page. But it also uses a bunch of other cool Fastly stuff (captcha, rate limiting, geo location, and more). Will share more in-depth later.

13

4 Comments

The USPTO just issued guidance on AI subject matter eligibility, part of their ongoing attempt to clarify exactly what is and is not eligible for patent protection, but this time focusing on AI inventions specifically. The guidance consists of 3 additional case studies, each with several claims directed to “AI” inventions, and each claim has its own subject matter eligibility analysis. You can find the examples here (https://lnkd.in/dQVKkRMQ) If you’ve ever found yourself unsure about whether or not the claims you are drafting are going to be rejected under 35 U.S.C. 101 as being “abstract”, don’t worry, because so has everyone else. It’s a confusing issue, thus why the USPTO has made special efforts to bring some clarity to this topic in the form of their 49 examples of subject matter eligibility analysis, each in a slightly different technological field, so no matter what type of invention you’re writing an application for, you can probably find an example that pertains to you. Of course, these 49 examples are spread across 200+ pages, in four different documents, each with different layouts, sounds inconvenient right? I thought so too, which is why I added a feature to IronCrow AI’s 101 Rejection Predictor to automatically find the most similar of these subject matter eligibility examples to each of your independent claims. I also added these 49 examples to IronCrow AI’s LLM Sandbox, as one of several “knowledge bases”, essentially making the LLM Sandbox a subject matter eligibility expert. Initial results are really impressive, amongst other things the LLM Sandbox can now instantly perform a full two-step analysis of the subject matter eligibility of any of your claims (or suggest amendments to make them more patent eligible), based on the full knowledge of all of the USPTO’s own examples of what constitutes eligible subject matter. As always, if you want to try out the 101 Rejection Predictor or the LLM Sandbox, just head over to ironcrowai.com and request a free trial. #PatentLaw #USPTO #AIPatents #SubjectMatterEligibility #Section101 #PatentEligibility #AIInventions #IntellectualProperty #PatentAttorney #LegalTech #AIinLaw #PatentDrafting #LegalInnovation #PatentExamination #AILegal #TechLaw #PatentStrategy #LegalAI #IPLaw #PatentAnalysis #LegalTechnology #AITools #PatentPractice #LegalIndustry #InnovationLaw #PatentProfessional #LegalSoftware #AILegalTech #PatentInnovation #LegalTechStartup

9

2 Comments

Project update! My application tracking extension is now doing fuzzier matching and injecting new information into LinkedIn (and other sites) to keep me from doubling up on applications. Next up: local storage and bulk import/export operations...

12

2 Comments

Its about solving the problem, not 'having AI' in the system. This is from last March, but the Wave 1 vs. Wave 2 distinction does seem to be playing out. Less 'generate me content', more 'synthesize vast amounts of relevant data and aid in decisonmaking or signal extraction'. https://lnkd.in/ehFBFC32 They don't talk about multi-agent systems here, but looking forward to more startups with a focus on leveraging agentic design patterns (https://lnkd.in/ed5rMmFe) to streamline real world workflows.

4

1 Comment

I recently read Tilde's blog post, it's a great deep dive into the cutting edge of sparse autoencoder research! They delve into the balance between interpretability and reconstruction accuracy, addressing issues like the "infinite width codebook" and "feature oversplitting" that often hinder robust feature learning. They also explore the Top-𝑘 activation function through the lens of Information Bottleneck theory, showing how it can improve robustness under noisy conditions. If you're interested in mechanistic interpretability or the theoretical foundations of machine learning models, I highly recommend checking it out! https://lnkd.in/eupt7JTs

16

For all those engineers out there building AI agents. You need to check out this #github repo. This is a great team tackling an important set of problems that engineers face, and this solution is significantly better than Langchain's approach. #ai #agents #ML

10

1 Comment

I am fascinated by what is happening under the hood of Large Language Models (LLMs). Earlier this week, Anthropic published a piece on mapping features in Claude Sonnet. The features (or concepts) were pretty abstract. But is it still pattern matching with vectors (a kind of SQL on steroids)? And if so, when concepts overlap and are scored based on the volume of the overlap, does the volume reflect the significance of the connection between them? 🤔 Read more here: https://lnkd.in/gBnJHdGM

13

2 Comments

I've just wrote a blog post on how I can see the AI moving forward next year, and how the Agent landscape will look like in the near future. The future with AI is wild. 🚀 https://lnkd.in/daHUYiBf

17