Daniel Ionut Vasile

To measure what I call Total Factor Reliability, we don't just ask: is Temporal robust? We must ask: do our *users* write robust Workflows? These improvements to Signals and Updates will help you ensure the latter. My blog, Robust Message Handlers, explains. We crafted them with love. Hope you enjoy!

7

1 Comment

LLMs and LangChain: The Dynamic Duo 🦸‍♀️🤖 Just finished a fun project where we built a Q&A bot using these two powerhouses. It's like having a personal AI assistant that's always ready to answer your burning questions! 🔥 Here's what I learned: LLMs are wizards: They can conjure up answers from thin air (or, uh, the internet). 🧙‍♂️ LangChain is the glue: It holds everything together and makes it easy to use. 🔗 Endless possibilities: This tech can be used for anything from customer support to writing catchy social media posts. 📝 Want to see it live (obvio a test server link 😛 ): https://lnkd.in/dZNAVCSV Note: it's a render link, they will take down the instance because of no use, you can hit the link and wait for two minutes, they will spin up the instance and you can try it out. Want to see the code? Check it out here: https://lnkd.in/dzjh4r47 #AI #MachineLearning #NLP #LangChain #LLM #Python #Developer #FunProject"

22

we're seeking feedback on our first open source contribution - Asimov, a high performance tool to build LLM apps, written in rust 🦀 we initially built asimov for our product at Overmind. link: https://lnkd.in/gbCYMJBv 1/ features an ergonomic, readable, WYSIWYG "what you see is what you get" api for prompt construction. it's super easy include your domain entities in the prompt. 2/ ships with an in-memory, wasm compatible vector db. also integrates with Qdrant 3/ chain models to build complex workflows (see comments for code) we use asimov heavily Overmind and will continue to iterate on it. currently, we're a small team but there's SO much to build - typescript and python bndings, a visual prompt editor to name a few. pls get in touch if you'd like to contribute ⭕

20

3 Comments

We've spent quite a lot of time improving Tapir's documentation lately: 1. we've got a new landing page - written with people yet unfamiliar with Tapir's and Scala's ecosystem in mind: https://lnkd.in/dkRT_kPT 2. there's a listing of how-to's, in the form of examples; each example is tagged, described, self-contained and easily runnable using scala-cli. Just copy the content of a single file! https://lnkd.in/dyJknFwx 3. finally, we've got a growing list of tutorials, each in a textual and video format: https://lnkd.in/eUKniQkT That way, we aim to address all 3 aspects of good technical documentation: tutorials, how-to's and reference documentation (that's what we mostly had so far). By no means "Tapir docs" is a done project, but we certainly got better foundations now! #Scala

70

In the AI ecosystem, LLMs act like CPUs, crucial yet dependent on complementary tools such as Chatbot and AnythingLLM, which function similarly to I/O systems. Alongside, tools like MemGPT and CrewAI serve roles akin to RAM and network interfaces. The challenge arises as these tools often evolve independently, aspiring to be standalone solutions. This siloed development leads to a lack of standardization, complicating their integration and limiting their utility in professional settings. Without unified protocols, integrating these diverse AI tools with LLMs demands significant customization, placing an undue burden on developers and hindering broader innovation. Just as the adoption of standardized interfaces like USB revolutionized computer usability, AI tools would benefit from similar standardization to enhance functionality and ease of use. Promoting common standards for AI tools is crucial for maximizing the capabilities of LLMs and ensuring their practical application across various domains. Without such efforts, the AI field may continue to struggle with fragmentation, undermining the potential of these advanced technologies.

3

2 Comments

The policies being driven forward by Thierry Breton around the EU AI Act have been utmost damaging for the European tech industry, and they need to be revised as soon as possible. We need to better balance driving European innovation through data collection and modeling with conflicting needs such as data protection and safety. We need to ensure we can turn these developments into businesses that can be the pillar of the next generation of companies that provide productivity growth in Europe. We have no other choice - either we do this here, or we will have to take over products developed in the US and in China, as it already happened with search engines, cell phones, and social networks. We wouldn't want to let this happen with AI applications in robotics or digital assistants. #euacc #euaiact #aiact

9

We're hiring ML Performance and Developer tooling enthusiasts at the Bay Area and London Google offices! When it comes to expensive ML workloads, every opportunity for performance counts.  Our Org’s mission is to accelerate the ML developer community, both at Alphabet and in the Open Source world, to quickly find performance insights. Are you excited about building developer tools - IDEs, frontends, snappy visualizations to be able to parse and efficiently present large volumes of connected data, in an intuitive way, that causes true user delight? Do you enjoy observing developer behavior and improving the e2e tooling experience to remove all points of friction along the way? Have you previously worked on the internals of frameworks like D3, Vis, Plotly? Are you looking to build company central and highly visible debugging tools used by a large number of internal and Open Source ML developers? Are you looking to make an impact at Alphabet scale by delivering performance opportunities across the company - ML Frameworks (JAX), ML Compilers (XLA), Model Owners, Product Area Leads, Senior Execs to guide business critical decisions? Are you passionate about bringing performance insights to life by building benchmark and visualization tools to reason about performance? Please reach out to me if you answered “yes” to any of the questions above, or have referrals so we can explore matching opportunities at Google. We’re looking for a wide variety of skills spanning background in Computer Architecture, Statistical analysis, Visualizing and presenting high quality insights, Frontend and Backend Engineering. Even if your ML background is limited, it is optional for many roles, and this is a great opportunity if you're looking for exposure to advanced ML development at Google. #DevCommunity #IDE #IntelliJ #OpenSource #vscode #Eclipse #xcode #DataVisualization #ChartJS #D3JS #VisJS #threejs #webdev #gamedev #indiedev #3D #WebGL #typescript #webassembly #wasm #JAX #TPU #GPU #TensorFlow #Numpy #PyTorch #AI #LLMs #Reasoning #MLPerf #mlplatform #deeplearning #hpc #BigQuery #DataAnalysis #Dataviz #jobalert #nowhiring #job #gethired #jobopening #jobfair #recruiting #jobopening #hiring #joinourteam #jobs #jobhiring #jobsearch #jobsearching #jobseekers #hire #opentowork #opentohire

46

3 Comments

We're seeing great momentum towards some roles. Please reach out if you have recommendations for exceptional Frontend Engineers (with some C++ background) and passionate about performance tooling. #DataVisualization #ChartJS #D3JS #VisJS #threejs #webdev #3D #WebGL #typescript #webassembly #wasm #angular #react #vuejs #javascript #frontend #visualization #figma #uxdesign #uidesign

13

1 Comment

𝐅𝐚𝐬𝐭𝐢𝐧𝐨 𝐞𝐱𝐢𝐭𝐬 𝐬𝐭𝐞𝐚𝐥𝐭𝐡 𝐰𝐢𝐭𝐡 $7𝐌 𝐩𝐫𝐞-𝐬𝐞𝐞𝐝 𝐛𝐚𝐜𝐤𝐢𝐧𝐠 𝐟𝐫𝐨𝐦 𝐌𝐢𝐜𝐫𝐨𝐬𝐨𝐟𝐭 𝐚𝐧𝐝 𝐈𝐧𝐬𝐢𝐠𝐡𝐭 𝐏𝐚𝐫𝐭𝐧𝐞𝐫𝐬 Latest blog post at Fastino dives deep into how AI is transforming the way we work and make decisions. Here are some key takeaways: 1️⃣ AI-Driven Efficiency: Discover how AI can streamline your operations, reducing costs and boosting productivity across your organization. 2️⃣ Enhanced Decision-Making: Learn how AI-powered analytics can provide deeper insights, enabling more informed and strategic business decisions. 3️⃣ Improved Customer Experiences: Explore the ways AI is revolutionizing customer interactions, from personalized recommendations to intelligent chatbots. 4️⃣ Future-Proofing Your Business: Understand why embracing AI now is crucial for staying competitive in an increasingly digital marketplace. 5️⃣ Ethical Considerations: We also discuss the importance of responsible AI implementation and its impact on your business reputation. #ArtificialIntelligence #BusinessEfficiency #AIInnovation #FutureOfWork #DataDrivenDecisions #CustomerExperience #AIEthics #BusinessAutomation #Fastino #TechTrends

1

LLMs in 2024 = endless prompting, inconsistent results, opaque APIs, runaway costs, and a GPU arms race 🤬 I’m excited to announce I've joined Fastino to help rewrite this equation 👆 Here's to a future with enterprise LLMs that get it right 🥂 (If you're building enterprise-scale AI systems and the above equation sounds familiar, drop me a note.)

105

15 Comments

The more research I read and the more I interact with LLMs, the more I wonder if some issues with combinatorial logic could be addressed by creating cyclic decoder-only models with a gating function for output. This approach might also enable a dramatic reduction in overall parameter count requirements for comparable performance. My understanding is that as model depth increases, higher-level abstractions form in the upper layers while some lower-level details also need to be propagated for use in the output token distribution. It seems larger models are effectively "unrolled" and need a fairly wide top to retain concept density as depth increases. By cycling a smaller graph, concepts and relations can intermingle more easily. If more layers of the smaller model are combined as input to generate the token distribution, they can specialize, reducing the need to propagate as much information up the graph for similar quality. At least, that's my hypothesis. The cycling could have either a learned gating function and/or a set number of cycles. I saw one research paper that seemed to touch on some of concepts, but I can't find it now. If anyone knows of other existing papers related to this idea, please let me know. If I've learned one thing, it's that there are almost no original ideas in model architecture. :)

9

Many folks often say that LLMs are just pattern matching… and contrary to some, I would actually agree with that sentiment! However, I really don’t think that scaling LLMs alone will help us breakthrough the next intelligence plateau. Surely, bigger models running faster and faster on smaller hardware will help.. but let’s take a moment to draw a comparison to human cognition and the brain 🧠 Pattern matching is the essence of reasoning but take the human brain for example - even though pattern matching is inherent to everything our brain does, we still have dedicated “modules” for various functions on top of pattern matching. And this is exactly the kind of system design we’re heading towards with agentic systems and cognitive architecture! Let's compare... > Prefrontal Cortex: Helps in analyzing trades offs, long-term consequences and planning complex actions (LLM Compiler/LDB, Self-Discover) > Hippocampus: Involved in forming new memories and integrating past experiences (stored as patterns) into current reasoning processes (GraphRAG + LGGMs + CLIN and other long-term memory systems for agents that use LLMs to pre-process, cluster and route to certain memories) > Basal Ganglia: Contributes to pattern recognition and procedural learning, helping automate certain processes based on recognized patterns (Automatic routing to SLMs, deterministic workflows and non-deterministic action planning) > Parietal Cortex: Integrates sensory information from different modalities, helping with spatial reasoning and the understanding of numerical patterns (multi-modal models, tools, environmental grounding and routing to prompting techniques specialized for math or other more nuanced capabilities) > Anterior Cingulate Cortex: Monitors conflict, makes decisions when there are competing patterns or information and adjusts strategies (Continuously learning models and systems inspired by CLIN and MedAgent-Zero + guardrails) > Default Mode Network: Active when the brain is at rest but also plays a role in reflective thought, daydreaming and consolidation of patterns into larger conceptual frameworks (Layered graph-of-thought consolidation in clusters / abstractions of thought with MCTS) I'm no neuroscientist so take what I say with a grain of salt... but I do think of LLM exploration of latent space and pattern matching as an essential substrate. However, it's the interplay between pattern matching and other cognitive processes that leads to authentic reasoning ✨ In 1-2 years time with enough speed + compute, these post training optimization techniques with environmentally grounded agentic systems will be what brings about authentic, human-like reasoning. And once these reasoning traces and agent trajectories get baked into the underlying models, this will only get better and better…

4

2 Comments

I recently found a nice study on code generated by large language models (LLMs), focusing on common errors and proposing solutions. Key findings: – LLMs generate shorter, more complex code. – Incorrect code often contains more comments. – As problem descriptions get longer, LLMs struggle more. – Misuse of APIs is a significant source of runtime errors. – Algorithm implementations are often suboptimal. – Closed-source LLMs generally outperform open-source ones. You can read the full paper at https://lnkd.in/d4HppCz3 I agree with the study's conclusions, and my experience using LLMs for coding aligns with its results (I don't usually use them because the benefits are outweighed by debugging time). LLMs can't fully replace a good developer yet but can help with simple tasks. They also offer a fresh perspective on your code, useful when colleagues aren't available for quick reviews. LLMs can sometimes quickly identify errors, acting as a substitute for Stack Overflow and Google searches. However, I wouldn't take LLM results at face value and would always double-check them. Who actively uses LLMs for coding? :)

1

How good are Trainium and Inference clusters for online serving of production traffic? Check this blog from Trishul Chilimbi on how Rufus responses are powered with high throughput and resiliency. #Trainium #Neuron #AWS Amazon Web Services (AWS)

6

𝗔𝗿𝗲 𝘆𝗼𝘂 𝗹𝗼𝘀𝗶𝗻𝗴 𝘀𝗹𝗲𝗲𝗽 💤 ? because you can't pinpoint your CI bottlenecks or spot problematic trends? At Pinterest, we faced the same challenges—but we’ve found a solution. 🌟 In our latest Pinterest Engineering blog post 🔗 https://lnkd.in/eJJ_GpGW We share how we’re leveraging honeycomb.io to identify CI root causes through cross-data correlations. Discover our innovative approach for categorizing CI failures in real-time using job log analysis. Learn how to segment CI jobs by agent wait time, script execution, specific Bazel targets, or any other level of detail tailored to your needs. Our enhanced CI observability metrics have let us improve CI stability and boost developer productivity—and they could do the same for you! Check out the blog for the full breakdown and see what’s possible when data-driven decisions guide your CI workflows. 🚀 Let’s make CI faster and more reliable together! #DevOps #DevX #CICD #honeycomb #observability #engineering #softwareengineer

16

2 Comments

ModernBERT is an open-source embedding model which represents a significant leap in encoder-only transformer architecture merging: > Long context (8k tokens) > Local-global attention for efficiency > Up-to-date code and text training data (2 trillion tokens) > FlashAttention & Unpadding to push performance on consumer and server GPUs > High retrieval and classification accuracy that sets new benchmarks across GLUE, BEIR, code tasks, and more. Learn more about this exciting new embedding model here: https://lnkd.in/en5ud9VE

2

Principal Engineer Staff Engineer Distinguished Engineer (my favourite) I’m going to say it and it’s likely a little rhetorical: I dislike all these terms immensely. Please can someone give me the canonical description for what they are supposed to mean? (I know I can use a search engine or AI to do it… I want to know what people think they mean). The industry I work in is in danger of disappearing up its own backside. What’s wrong with: Software engineer Why do we need to differentiate? The only reason I can see is signposting to people with differing levels of experience or different sets of responsibilities. If we need to do that then, I’m not sure our teams are set up correctly in the first place. #software #softwareengineering #softwaredevelopment #productdevelopment

97

104 Comments

A blog on (very) nerdy topic that I worked on with our (amazing) intern Xiangpeng Hao this year: https://lnkd.in/eQDfvNfH. We quantify how much of the discussion about changing Apache Parquet metadata was shiny-new-object syndrome and how much was actual technical limitations. We think there is easily a factor of 4 to be had from existing implementations before new formats are needed

290

12 Comments