Nora Hong

Recording of last night’s fireside chat with Kanjun and Anyscale CEO Robert Nishihara up now on our YouTube! https://lnkd.in/gDu6gyZd Cannot say thank you to Robert enough, who on just a few hours’ notice agreed to move up his fireside chat by several weeks after our original speaker had to postpone. Extremely fun and expansive conversation. Topics covered: 01:18 Robert's founder journey 05:43 What kinds of open-source projects can make good business 08:17 How the AI boom changed Anyscale’s business 10:00 Anyscale's new LLM API 11:05 How enterprise companies use Anyscale 12:50 How startups use Anyscale 15:29 Agents 17:20 Anyscale Endpoints and Anyscale Private Endpoints 18:04 Why companies don't want to give their data to OpenAI 19:45 How Anyscale offers the lowest price point on the market:  $1 / million tokens for Llama-2-70B 20:40 Anyscale's new LLM-powered features 22:36 The future of open-source 23:24 The future of AI, and whether Robert’s vision for the future when he first started Anyscale came true 27:12 "Under-hyped" fields in AI 29:59 The GPU scarcity, NVIDIA, Databricks, how Robert would build an AI company if he were to start fresh today, and more incredible audience questions

34

We find it encouraging to see the metrics that Waymo and Google are showing with regard to accidents. But are they the right metrics? Keeping people safe is key, but the fundamental challenges of mobility are optimization, orchestration, reducing emissions, and addressing Vision Zero. The root challenges can only be addressed with Intelligent Infrastructure. Trying to deploy autonomous systems without INTELLIGENT INFRASTRUCTURE is like trying to build the internet without networks. Texas, the State of the Future, is executing large-scale Intelligent Infrastructure deployments today to save lives and reduce economic losses. The Federal Infrastructure Bank and infrastructure investors are mobilizing billions to underwrite large-scale deployments. https://lnkd.in/gWnJD877 Autonomy Institute https://lnkd.in/gtwx9BnH #infrastructure #future #transportation #arpa-i #arpax #jobs #bank #digital #innovation #data #people #visionzero #continuumofcapital #dataexchange #partnerships #supplychains #digitaltwin #privacy #investment #p3programs

15

2 Comments

Fascinating talk by Anyscale CEO Robert Nishihara, at Thursday Nights in AI with Ali Rohde. AnyScale also demonstrates how A.I. is also intersecting with so many adjacent businesses, apps and efficiency challenges in the era of open-source and foundational models. Robin Singh Products like Ray and Anyscale Endpoints seem really salient. Outset Capital is a pretty fascinating entity and I'm really enjoying these talks. They even have their own Newsletters: https://lnkd.in/gntcEd3d Anyscale as a startup has a lot of Databricks vibes and utility. Can't wait to see what they become.

1

5 years ago today, I finally found “It”. It sure took a while though… After starting my career at PwC in 2008, and then spending nearly a decade in medical device sales, I felt lost and somewhat hopeless. I’d been chasing what I thought I was supposed to do, rather than what I enjoyed doing. That all changed on Nov 20th, 2018. An example of the beneficial side of social media; a targeted ad for Codecademy, called Introduction to Python. The aforementioned “It.” My knee-jerk reaction was “no way, not happening.” I was 34 years old at the time, so the thought of learning how to code seemed both daunting and silly. And the thought of changing my career path… well, that wasn’t even on the radar. Still though, my mind was open enough to it… Why?... Desperation. The feeling of being lost and hopeless is both awful and terrifying. I’m sure many can relate. When you’re in that place though, opportunity is lurking… The opportunity to dig out... or, the opportunity to spiral further down. To be clear, this post isn’t meant to give off “pull yourself up by your bootstraps" vibes. When it comes to athletics and physical activity, I'm for that mindset (most of the time). However, when it comes to the mind, human emotions, and the unique, singular life experiences we all have, there is no simple, cookie-cutter statement or approach to “fix things” – in my opinion. And if you believe there is, congratulations on figuring out what works for you. Just keep in mind that’s a gift, not someone else’s defect. To that point though, this post is about how grateful I am, on a few fronts – and how I hope to convert that gratefulness into becoming more of a contributor, rather than just an absorber. I’m grateful that I found Python. Grateful that I discovered it in an era where I could learn an immense amount through very inexpensive, highly valuable platforms such as Codecademy, Coursera, Dataquest, Udemy, and many others. Grateful towards the amazing Python community, and community of coders overall, which allowed me to quickly build a full-stack skill set. Lastly, but certainly not least, extremely grateful that ChatGPT and then GPT-4 came along when they did. So today, I'm reflecting on my gratitude and my desire to contribute more. I've used Python to delve into a plethora of interests - sports, politics, pandemic data, and dozens of other topics/areas, which I’ll be sharing daily (hopefully). Starting with my GitHub repo that makes use of Spotify's API (and the excellent Spotipy Python library). A great API to tinker with. Tinkering only leads to growth, and growth compounds over time - and there's no requirement for every pursuit to generate revenue or have an end goal; just learning in and of itself can be the purpose. Selfishly, I know I will learn a great deal from this experiment too. Autonomous_AJ, signing off (beep boop beep) Link to GitHub repo: https://lnkd.in/gZY4SDYr

1

I'm thrilled to announce that our project SafeGuard, built in collaboration with Chuyi Shang, Aryan Goyal, and Siddarth Ijju, has been awarded the "Best Overall Generative Application Built on Amazon Bedrock" at the UC Berkeley AI Hackathon 2024! 🏆🚀 Inspired by the increasing threat of prompt injection attacks, we developed SafeGuard, a robust Python package designed to intercept prompt injection attacks on LLM applications. By only editing 3 lines of code, SafeGuard allows you to wrap and protect any language model call. SafeGuard employs a custom-trained detection model that outperforms the most popular models while being twice as small. Our modular, easy-to-use solution ensures safer deployment of LLMs by classifying and sanitizing potentially harmful prompts, providing a reliable shield against various injection threats. The event was an incredible experience, featuring an inspiring speech by OpenAI co-founder Andrej Karpathy. A huge thank you to the organizers—University of California, Berkeley, Cal Hacks, Berkeley SkyDeck, Amazon Web Services (AWS), and all sponsors—for their support. Special thanks to Brandon Middleton, the rest of the AWS team, and the judges for your valuable time and feedback! We’ve released the open-source code for our detection pipeline, model training, and synthetic data generation. We will also upload the package to PyPI as an installable package after cleaning up the code. If you find any of this interesting, we welcome contributions! We hope this can become a useful tool for many. 🔗 Links: * GitHub: https://lnkd.in/gc2z8gub * Devpost: https://lnkd.in/g3HZJWB8 * Model: https://lnkd.in/ge5Y2gg3 * Dataset: https://lnkd.in/gPQ75_ER #BerkeleyAIHackathon #CalHacks #Skydeck #Hackathon #AI

108

18 Comments

I've been thinking for a while about hosting an in-depth technical class in Generative AI and LLMs to help interested folks get started building and deploying LLM applications no matter where they are in their career. I have decided that August will be a perfect time for this. It's going to be a combination of online and physical sessions for up to 4 weeks. These sessions will connect learners to some of the best engineers and practitioners in Generative AI locally and internationally. Here is an initial breakdown; I will add to this as new ideas come along. Theme: AI Summer of Code Focus Areas: 🔸 RAG 🔸 Evals 🔸 Agents 🔸 Web Search 🔸 Multimodal 🔸 Finetuning 🔸 LLM-Local 🔸 LLMOps 🔸 Foundations (a little theory?) 🔸 Data Engineering for LLMs Target audience: 💼 Beginners, data professionals, and developers / engineers at any level looking to learn and master practical applications of LLMs. Prerequisite: 👨🏽‍💻 Familiarity with a programming language such as Python. Program Mode: 🛜 Online, 4 weeks (free): recorded lessons, tutorials, live sessions, guided projects 📍 Physical, 4 weekends (paid): hands-on tutorials, workshops, expert talks. The physical sessions will require a small fee commitment so we can provide the best experience for learners. It will have all the benefits of the online sessions + networking, one-on-ones, exclusive sessions, etc etc. The physical sessions will hold in Lagos. Both the online and physical sessions will be intensive and hands-on. No long talks, no BS, just straight up head-first dive into code and best practices + expert talks to gain deeper insights. I want this to be a rewarding learning experience for participants, not just in terms of added knowledge but also in career prospect. To achieve this, I'm putting together a lineup of AI experts and experienced engineers which I will be announcing soon. As a participant you will learn from the best and build fun projects you will be proud of. So if you are getting started in your Generative AI journey or want to further your skills with hands-on projects, this is for you. Apply here: https://lnkd.in/dK8GtH8s I'm also calling out for volunteers. If you would like to support on one or more aspects of the program, please fill the form here: https://lnkd.in/di6JK7Yz Who would you like to see attend this program? Tag them in the comments below 👇🏽 Let's spread the word 😉 cc Ayo Kehinde Samuel #AI #genai #llm #MLOps

132

38 Comments

Sharing an exciting story from Zoom’s CTO, Xuedong Huang, on the company’s federated approach to AI. It’s allowing them to unite diverse AI services in order to enhance communications and improve quality & response times, while lowering costs. Their Zoom AI Companion leaps ahead by integrating multiple languages and offering premium AI benefits, at no additional costs to users.    We at Hitachi are also extremely interested in this approach and incorporating it in several key areas. For example, our Hitachi Digital Services group is augmenting a federated approach with a multi-agent architecture within its Hitachi Application Reliability Center services that allows us to achieve maximum benefits at lower costs.   Another great example of the fast pace of AI. https://lnkd.in/eZqsVaZb Premkumar Balasubramanian Frank Antonysamy Roger Lvin

48

1 Comment

Accelerated computing architecture is driving the future 🙏💜🙏🧿

3

Imagine having an AI assistant within your IDE, understanding your codebase and project context. That's #AmazonQ Developer. Here's a cheat sheet from Brooke Jamieson to help you get the most out of this powerful tool: https://go.aws/3X3NpAc Quick Commands: /clear - Start fresh with a new chat session /dev - Get an implementation plan for new features /transform - Upgrade to a new code language version /help - Overview of Amazon Q's capabilities In-Line Code Assistance: Explain - Get plain language explanations Refactor - Improve code readability and efficiency Fix - Debug issues in your highlighted code Optimize - Enhance performance with AI optimizations Send to Prompt - Ask follow-up questions about your code Example Prompts: "What AWS services for a serverless backend?" "Explain this React component" (Send component) "/dev Add user authentication to my app" "How do I test this Python function?" (Send function) "/transform my Java 8 project to Java 17" "Fix: Why is this code throwing an error?" (Send buggy code) "Best practices for DynamoDB data partitioning?" Amazon Q Developer contextually understands your project to provide tailored guidance, explanations, and direct code improvements - all within your favorite IDE. Put this AI pair programmer to work: ask questions, send code snippets, and leverage unique capabilities like code transformation. The more you engage, the more value you'll get! #AWS #CloudComputing #AI #generativeAI #AmazonQ #SoftwareDevelopment #Coding #IDE

288

4 Comments

Anyscale, the AI infrastructure company, launches a new service, Anyscale Endpoints, which allows developers to integrate large language models (LLMs) into their applications with popular LLM APIs. It's a cost-effective, scalable solution that's up to 10 times less expensive for specific tasks. https://lnkd.in/dT2WJvnj #aiinfrastructure #integratedsolutions #apiintegration #infotechreport #media7

3

New Case Study 📈 Why Topo.io (YC W24) Switched From Amazon RDS to Neon "We wanted to do something scalable...while still being able to iterate fast and have a high velocity." Learn more about how serverless, branchable Postgres enables Topo to iterate fast without compromising reliability → https://bit.ly/4bq9fCK #postgres #postgresql #casestudy #rds #ycombinator

12

In between attacking Thanksgiving leftovers, I spent my weekend getting familiar with Open AI's Whisper API. If there's one thing I hate about interviewing someone, it's transcribing them afterwards. So I figured this would make an excellent use of Whisper API's amazing speech recognition capabilities. It took me a few hours, but I finally managed to fiddle around with Python and get Whisper API working on my machine. Then I learned someone else had already done all that hard work for me. Developed by André-Abush Clause, the Open AI add-on for NVDA allows screen reader users to use Chat GPT, Whisper API, and the Vision API all with one or two keystrokes. Users can even upload an audio file directly and the API will transcribe it. All without any Python knowledge. I've been using the Vision API to take screenshots of popular websites and getting a visual understanding of their layout and design, something I have never had unfettered access to before. The Vision API has also helped my confidence with taking screenshots and I may start posting more of those here for you all. I'll be spotlighting some more useful NVDA add-ons this week, so follow me to learn about NVDA and web accessibility as a whole, with a little AI geekery sprinkled on top. #NVDA #AIAccessibility #WhisperAPI #VisionAPI #blindnessawareness

2

I am both excited and proud to share that our team has just launched our containerized deployment of the ComputeAI SQL engine for data lakes on github (https://lnkd.in/g5ziEceS). The containers should work on most versions of Linux and also desktops (though a desktop specifc version is in the works). The project required an incredible amount of determination and dedication from our 4 coders over a period of 2 1/2 years, and the project would not have been possible without the support and belief of our investors in what we are doing (https://lnkd.in/gGU-qv93). With our product we hope create immense value in the open Spark SQL (Presto/Trino SQL support is in prototype phase) ecosystem to create unlimited concurrency at high performance, reliability, and efficiency while reducing cloud infrastructure cost by ~5x. Here's our value prop: -Massive concurrency with unlimited users/queries -Extreme reliability (no OOM-kills) -Multi-cloud (AWS, Azure, GCP), hybrid, and on-premises -High level of cloud security with enterprise grade SLAs -100% open standards (Parquet, Iceberg) -Cluster up and running in less than a few minutes -No touch, requires no management or tuning Simply download the container, develop using GBs of data, and run 1000s of concurrent jobs with TBs of data. There is no management or tuning needed. ComputeAI runs reliably and unattended like an OS. If you want to run elastic clusters (also free) then reach us at contact@compute.ai and we will send you the scripts. It literally takes 5-8 minutes to create a cluster (a one time thing since everything auto shuts down when not in use). To find out more, please check out our website at https://compute.ai/ If you like the sound of this and are interested in joining our team, please reach out to us at contact@compute.ai. We'd love to chat! We're a small team of 4 developers and always looking to work with other talented and fun people. Our goal is to give back to the open source community. We recognize that we are standing on the shoulders of giants with over a decade of work put in by open source developers. With our product, we hope to create incremental but significant value in the world of SQL based compute directly on files and external tables. No more OOM-kills or over-provisioning of memory. That is our primary offering. We are not the fastest distribution but we are not terribly slow either :). We will continue to share our performance numbers as we make more progress. However, we do offer some of the highest concurrency for complex SQL. So please put these bits to the test and share your thoughts with us. Roadmap item: Containers that run on your desktop. This already works in the lab but we need to productize this. We also run on ARM and Raspberry Pi and are edge ready. Lastly, if something doesn't work or if you have any questions, please email us at contact@compute.ai. We are more than happy to help. #parquet #iceberg #oss #analytics #datawarehouse #datalake

100

42 Comments

Anysphere, a two-year-old startup that’s developed an AI-powered coding assistant called Cursor, has raised over $60 million in Series A financing at a $400 million post-money valuation Read below to find out more about it #tech #ai #github #webdevelopment #technology #article

2