Hey nice - this stuff is so much fun to me. I've worked a number of experiments like this too, especially related to live coding. Love seeing it in the wild.

I built a small project where you can live-code Love2D. The running program updates in real time (no saving needed) and see all values update in real-time, via LSP.

https://github.com/jasonjmcghee/livelove

And also added the same kind of interactivity like a number slider and color picker that replace text inline, like yours (though via vs code extension: https://gist.github.com/jasonjmcghee/17a404bbf15918fda29cf69...)

Here's another experiment where I made it so you could "drag and drop" to choose a position for something by manipulating the editor / replacing a computed position with a static one, on keypress.

https://clj.social/@jason/113550406525463981

There's so much cool stuff you can do here.

That's cool! I suspected that someone must have done something like this before.

I love your project. I am making something with LSP and your code was a great example of what is possible.

Glad it was useful! LSP is definitely under-utilized. It's great for the "malleable software" / small tools for yourself kind of stuff especially.

very popular on tech twitter. Right up there with "we're back" and "we're so back"

I feel like they've really changed how they are presenting themselves but Runway was the big one that came to mind for me that focus on more than simple video generation but actually meant for film makers.

This is the best page I found- maybe there's a better one...

https://runwayml.com/product

But its interface looks like a video editor- timeline, etc

Nice project! A regular on HN and creator of usearch built an embedding search for the same dataset and did a write up which is a great read.

https://ashvardanian.com/posts/usearch-molecules/

Thanks — I read Ash’s post (great blog!) and even spun up USEARCH when I first explored this space.

Main differences:

* *Cost-efficiency:* USEARCH / FAISS / HNSW keep most of the index in RAM; at the billion scale that often means hundreds of GB. In CHEESE both build and search stream from disk. For the 5.5 B-compound Enamine set the footprint is ~1.7 TB NVMe plus ~4 GB RAM (only the centroids), so it can run on a laptop and still scale to tens of billions of vectors. This is also huge difference over commercial vector DB providers (pinecone, milvus...) who would bill you many thousands USD per month for it, because of the RAM heavy instances.

* *Vector type:* USEARCH demo uses binary fingerprints with Tanimoto distance. I use 256-D float embeddings trained to approximate 3-D shape and electrostatic overlap, searched with Euclidean distance.

* *Latency vs. accuracy:* BigANN-style work optimises for QPS and milisecond latency. Chemists usually submit queries one-by-one, so they don’t mind 1–6 s if the top hits are chemically meaningful. I pull entire clusters from disk and scan them exactly to keep recall high.

So the trade-off is a few seconds slower, but far cheaper hardware and results optimized for accuracy.

Heads up, the token cost breakdown tables look white on white to me. I'm in dark mode on iOS using Brave.

Should be fixed now. Thank you!

I can't tell if this is a meme or not.

And if someone had this idea and pitched it to Claude (the model this project was vibe coded with) it would be like "what a great idea!"

If the author stops by- the links and the comments in the code blocks were the ones that I had to use extra effort to read.

It might be worth trying to increase the contrast a bit.

The content is really great though!

Also very curious about what kind of model this is and how it could (so far as it sounds) take 100% of the hardware for 15 seconds per request.

llama.cpp is optimized to serve one request at a time.

vllm is optimized to serve many requests at one time.

If you were to fine tune a model and wanted to serve it to many users, you would use vllm, not llama.cpp

Here's a super relevant comment from another post https://news.ycombinator.com/item?id=44366418

Appreciate it!

> the models I do use remain static and predictable

Some people overload "local" a bit to mean you are hosting the model - whether it's on your computer, on your rack, or on your hetzner instance etc.

But I think parent is referring to the open/static aspect of the models.

If it's hosted by a generic model provider that is serving many users in parallel to reduce the end cost to the user, it's also theoretically a static version of the model... But I could see ad-supported fine tunes being a real problem.

I am bewildered to hear the sense of "local," in which I and engineers in my experience have for thirty years referred to things which are not remotely hosted, referred to as an "overload."

My definition of local is the same as yours. It's just that my mind got focused on the open models part and I forgot that you specifically mentioned a "local" setup. My bad

I thought it was friendlier than "abuse"

Less absurd, anyway. In what sense does one "abuse" a word to use it in its dictionary definition? You appear to be a good example of where this benighted excuse for an industry is headed. Explain it to me.