Jim Gao

What an event (based architecture) 🔥 We’re thrilled to share a new article by Yossi Rizgan, Engineering Group Lead at Gong, where he explores the architectural patterns and core building blocks behind our scalable, low-latency, and reliable communication capture systems. Dive into the details and discover how we’ve built these powerful systems!  Read the full article here 👉 https://lnkd.in/e2yXA2iD

10

Amen Daniel Pope, and this piece nails it... "While DLC effectively cools AI chips, it does not eliminate the need for substantial air cooling. This dual requirement means that facility water production plants must still produce water at temperatures of at least 15-17°C. Additionally, servers, rear door heat exchangers, and air handling units with CRAH (Computer Room Air Handler) units still rely heavily on fans. This setup relies on the use of chillers or evaporative cooling to maintain optimal temperatures, which in turn prevents significant reductions in Power Usage Effectiveness (PUE) and Water Usage Effectiveness (WUE)." #digitalinfrastructure #liquidcooling #immersioncooling

36

6 Comments

Second article in the propublica/Seattle times series on data centers in the state of WA. This one digs into the tax breaks and subsequent lack of accountability in around jobs created etc (that was the some of the rationale for the original tax breaks). Basically the tax breaks have effectively turned into a giant giveaway. https://lnkd.in/gPFZgqvq

1

More than 3.5GW of data center capacity is due to come online in California between now and 2029. Around a third of the requested capacity was from one customer. First reported by DCF, a June investor update from Pacific Gas and Electric Company (PG&E) outlines the massive data center development pipeline for northern and central California. In the update, PG&E notes that the areas around San Francisco and Silicon Valley – long one of the largest data center markets – continue to see high demand. In San Jose, more than 1.6GW of data center capacity has been requested from 2025 to 2029 from 11 customers. These range from 10MW up to 600MW. Around 100MW of capacity has been requested for 2025 alone.

1

The future of vector embedding storage is integrated, not isolated 🙌 Learn how new solutions are addressing traditional vector database challenges by treating embeddings as indexes. [Timescale Weaviate Pinecone LanceDB] 📈 https://lnkd.in/gRQg7Z_8

19

1 Comment

As the digital world expands, the need for environmentally responsible data centers has never been greater. I'm excited to share insights from my latest article, "Nature Positive Data Centers – Playing The Infinite Game," where I discuss how the data center industry can lead in environmental stewardship. We're at a pivotal moment where integrating sustainability deeply into our operations can significantly impact our planet's health. In the article, I discuss the shift from traditional metrics to holistic ecosystem impact quantification, a practical to drive data centers towards becoming virtually invisible in their ecological impact. This transformation is not just about reducing our footprint but enhancing the ecosystems we inhabit. Join me in driving a sustainable, thriving future by embracing innovative, nature-positive solutions. Read the full article here:  https://lnkd.in/g5G-9JxP #DataCenters #Sustainability #Innovation #NaturePositive @Ecosystem Intelligence @Sean James @Kaitlin Chuzi

140

19 Comments

Big tech on track to pour more than $180B into data centers this year Summary: In the third quarter of 2024, leading technology companies significantly increased their investments in data center infrastructure, collectively surpassing $180 billion in capital expenditures for the year. Amazon, Google, Meta, and Microsoft together reported an 81% year-over-year increase in infrastructure spending during this period. This substantial investment surge is primarily driven by the escalating demand for AI model training, which requires advanced and expansive data center capabilities. Major cloud service providers—AWS, Azure, and Google Cloud—each reported significant year-over-year increases in both capital expenditures and AI revenues in Q3, with projections indicating continued growth in the coming months. The focus on AI-optimized infrastructure reflects a strategic shift to accommodate the computational demands of AI applications, underscoring the critical role of data centers in supporting the evolving landscape of cloud computing and artificial intelligence. #DataCenterExpansion #AIInfrastructure #CloudComputing #TechInvestment #AWS #Azure #GoogleCloud #ArtificialIntelligence #CapitalExpenditure

15

1 Comment

if you know, you know (ai) " Data centers take time to build and outfit. Chips face supply shortages and costly fabrication....wiggle room...infrastructure, if the future turns out to be not exactly what it expects. " there's a horse race. just ask the magnificent 7. each has taken a position on the roulette table. carre. square. it's a term used when you bet on the table and place chips in the form of a square. in other words covering a small corner of the game. in anticipation that probability will play itself out. this is a bit of what's happening in the game of data centres. Equinix is rarely mentioned although they are there. the usual suspects Google Microsoft Amazon now NVIDIA. there is a strategy being played out. it is to be the computer of the world. last year's Microsoft keynote with satya said as much. i guess it's a natural extension of bill gate's gated gate of scarcity in the form of a computer on every desk, in every home. well we have that in a way. and thanks to Apple it's in our pocket in the form of iPhone, rn. so the natural extension is to be the 'cloud' or rather the data centre that you as an (ai) startup are ported on top of this hardware. hmmm. so what's next? well this article points to the investments. billys being peppered to the right to the left. it's a game of brute force. hahahahaha. ummm. reinforcement learning applied to financial investment. funny how there is so little imagination amongst the smartest people in the room. so even they in the highest corners of this horse race are (un)certain as we are in a moment of (r)evolution. who wins? well it's supposed to be a selection of the (ai) tech founder, the team, the application developers. we have some insight here from the eggheads. a16z. dude they are in the middle of this too. worth a watch. https://lnkd.in/eYwMQfcc so good that they share their thesis. of course they are talking their book. gotta keep being on top of the $ directed to them to decide where, what, when. that's the arc. a bag of money as in a seed that grows and the crop is shared. i mean. okay. whatevs. i've been writing that the next 10 years will define the next 1000 years. after a looksy around sv i've changed this. it's 5 years. sounds a bit alarming. although given the investment capital this is craving a omnipotent device in the form of (ai) to accurately know what is (un)known to the wider audience. it's the same old same old of rent-seeking. we are the landlord. where are the rentees. what can they afford as a take rate. what? let's price this accordingly so that we do not kill the virus, as the host. we know that (ai) is designed to perfectly be created so that it could kill the virus. in this moment then well the game ends. and it sucks to be alone. https://lnkd.in/eHVd_f_Y boring isn't it. source: https://lnkd.in/eJC_HGzh < . >

2

1 Comment

The legacy data center revolution How innovative strategies can transform data center infrastructure, enhance efficiency, and unlock potential in healthcare and finance Summary: The legacy data center landscape is undergoing a transformation driven by the rising demands of AI and high-performance computing (HPC). Traditional designs, characterized by raised floors and low power densities (8–10 kW per rack), are being replaced by next-generation infrastructure capable of supporting up to 100–240 kW per rack. These innovations improve efficiency, reduce space requirements, and unlock the potential for AI applications in critical industries like healthcare and finance. Key Points: Challenges of Legacy Data Centers: Older designs, such as raised floors, are inadequate for modern AI workloads due to weight limits and cooling inefficiencies. Power distribution and cooling systems must evolve to meet the high-density requirements of AI and HPC workloads. Future-Proofing Strategies: Transitioning from 100 racks at 10 kW each to 10 racks at 100 kW each significantly reduces IT floor space while increasing processing power. Innovations include modular, prefabricated systems with remote monitoring and liquid cooling for energy efficiency. Data centers of the future will feature less internal IT room space, with externalized cooling equipment like chillers. Applications in Key Sectors: Healthcare: AI can process vast amounts of medical data to identify trends and improve diagnostic accuracy, enabling tailored treatments. Finance: AI enables personalized financial advice and predictive analytics for better decision-making. AI Integration Models for Legacy Centers: Off-the-shelf AI models from cloud providers for quick deployment. Private cloud environments for customizable, energy-efficient AI solutions. In-house AI development for tailored, application-specific optimization. Operational and Cybersecurity Considerations: Legacy facilities must integrate robust power and cooling systems while addressing vulnerabilities like unsecured IoT devices or air conditioning units. Advanced DCIM software ensures up-to-date encryption and secure infrastructure management, reducing cyberattack risks. Implications for the Industry: Efficiency Gains: Transitioning to high-density designs reduces costs, improves energy efficiency, and optimizes floor space. AI Readiness: Modernized data centers support AI applications, unlocking potential in sectors like healthcare and finance. Cybersecurity Priority: Safeguarding critical systems and minimizing vulnerabilities are essential for sustainable growth. #DataCenters #LegacyInfrastructure #AIIntegration #HighPerformanceComputing #EnergyEfficiency #LiquidCooling #CyberSecurity #HealthcareAI #FinancialAI #FutureOfIT https://lnkd.in/gi-FVNuq

6

New Data Centers are coming up all across the U.S. and they need power as well as water. As per a recent report from the Electric Power Research Institute (EPRI): ✅ Fifteen states accounted for 80% of data center load in 2023: Arizona ranked sixth. ✅ Data centers accounted for 7.43% of Arizona's total electricity consumption in 2023. ✅ Based on different growth scenarios, this could increase to anywhere from 8.81% to 16.58% by 2030. Arizona has abundant sunshine (solar energy) and has a low risk of natural disasters. However, water can be a challenge. More power, more water, more challenges for the local utilities.... Arizona Public Service - APS Salt River Project #sustainability #datacenters #ai #water

7

3 Comments

It sounds like they need a #datacenter solution with better usage of power and land, with, ironically, no sound. Sounds like #immersioncooling 🤫

4

Well said Charles Giancarlo - "Most people assume that data centers are entirely based on modern technology. However, in 2024, over 80 percent of digital information still sits in data centers on spinning mechanical hard disk drives (HDDs). This energy-intensive technology, celebrating its septuagenarian anniversary, requires enormous amounts of power, cooling, and space and fails more often than currently available technology... We must demand more efficient data centers or else their continued growth will place an unsustainable strain on energy resources, impact new home building, and increase both carbon emissions and California residents’ cost of electricity." #sustainability #energy #AI #electricity

34

Did you know that new federal energy standards are on the way for commercial buildings? The updated ASHRAE Standard 90.1 is intended to boost efficiency and energy savings for commercial buildings. U.S. Department of Energy (DOE) projects up to 14% site energy savings, 14.7% source energy savings, and 14.7% carbon emission cuts as a result of the new codes. For the first time, the standard includes on-site renewable energy systems. The criteria for building owners relate to insulation levels, lighting power, HVAC system controls, and whole-building performance. States have two years to meet or exceed the new codes. Read more via Industry Dive below: <https://lnkd.in/g6Up3QN9 And learn more about how Redaptive can help you stay up-to-date with the latest energy efficiency requirements at <https://lnkd.in/gYgeBYX3 #decarbonization #netzero #energyefficiency #buildings

4

1 Comment

"Data centers in space" is a concept that could actually work, study finds https://zurl.co/lb1V #ai #genai #datacenter

1

"What We’ve Learned From A Year of Building with #LLMs" A really informative post from Amazon's Eugene Yang and other #AI luminaires. A long post, split into three section, which I would recommend reading end-to-end. Here are some of the key takeaways: 💡 Tactical: Nuts & Bolts of Working with LLMs ➡ Focus on fundamental prompting techniques such as n-shot prompts, chain-of-thought, and providing relevant resources ➡ Structure inputs and outputs to help models understand the task and return reliable output ➡ Create small, focused prompts that do one thing well ➡ Carefully craft context tokens and challenge assumptions about how much context is needed ➡ For retrieval-augmented generation (RAG), the quality of output depends on the relevance, density, and detail of retrieved documents (totally agree !) ➡ Don't forget about keyword search and use it as a baseline and in hybrid search approaches ➡Prefer RAG over finetuning for incorporating new knowledge ➡ Long-context models won't make RAG obsolete ➡Use multi-turn flows and step-by-step approaches for large performance boosts ➡Finetune models only when prompting and other techniques fail ➡ Create assertion-based unit tests from real input/output samples ➡ LLMs will return output even when they shouldn't, so robust guardrails are needed ➡ Hallucinations are a stubborn problem that require a combination of prompting and guardrails 💡Operational: Day-to-day and Org concerns: ➡ Review samples of LLM inputs and outputs daily to identify issues quickly ➡ Migrating prompts across models is challenging and may require re-engineering ➡ Choose the smallest model that gets the job done ➡ Involve design early and often in the product development process ➡ Design the user experience for human-in-the-loop feedback and data collection ➡ Ruthlessly prioritize the hierarchy of needs based on what's essential for the product ➡ Calibrate risk tolerance based on the specific use case and audience ➡ Focus on the process of working with LLMs, not just the tools ➡ Always be experimenting/prototyping and set aside time for it during planning (agree, I am a big fan of prototyping) ➡ Building AI products requires a range of specialized roles, not just AI engineers 💡Strategy: Building with LLMs without Getting Out-Maneuvered: ➡ Training models from scratch almost never makes sense for most organizations ➡ Don't finetune models until proven necessary through collected examples ➡ Start with inference APIs but consider self-hosting for control and cost savings ➡ Focus on building a system around the model for lasting value and differentiation ➡ Don't build LLM features that can be bought off-the-shelf ➡ Center humans in the workflow with AI as a supporting tool, not a replacement ➡ The cost of LLM capabilities is rapidly decreasing, so plan for future affordability ➡ It's time to move from one-off demos to scalable, reliable products #EnterpriseAI #AIBestPractices

5

Raises a few questions around carbon-free vs sustainable, and the various costs and benefits of AI. Two things come to mind for me: 1) A discussion with CIOs on the value of classic ML vs GenAI for industrial use cases. Many of the CIOs I talked to saw the larger benefit in complex optimizations to drive efficiency, predictions and smarter planning. Most of these cases were not GenAI but a combination of machine learning a deep understanding and modeling of the respective industry domain. Yes, there are some excellent GenAI cases, but it seems to be worthwhile to choose the best (and not always the most shiny) tool for the purpose. 2) The true cost of energy: I am clearly not the expert here, reading the DB Research paper comparing the overall cost (decidedly a non-leftist activist point of view), and wonder whether nuclear for AI is the smartest choice. Will post the link in the comments. I have mixed feelings here, honestly. As a tech product manager, I celebrate the potential efficiency gains and the completely new possibilities. An an environmentalist and engineer, I would love to see the overall ‚energy balance‘ of the system.

16

1 Comment

Forward thinking executives recognize that AI will drive demands for Data Center, and it in turn is limited by power. Hence the investment being made by some. What will be impact on the cost of using AI if the rates increases due to shortages? Something to consider

8