I made an open source app to rewind & search everything happened on your screen (on Windows)
Made with
Page icon

I made an open source app to rewind & search everything happened on your screen (on Windows)

April 21, 2024 5:18 PM
MyWork
1 more property
An Open Source Mac App Rewind / Copilot Recall alternative tool on Windows to help you retrieve memory cues.
Perhaps you've had moments like these: you recall reading or seeing something, but when you try to retrieve it, you search through every app in vain. Particularly with information dispersed across multiple web pages, videos, and interspersed chat messages, they appear and disappear in the blink of an eye. When you try to dig back through your memory or browser history to find it, it's like it was never there expect your imagination. Worse still, they may been deleted, hidden, or modified.
Black Mirror S1E3 "The Entire History of You"
ALT
As someone challenged by keeping memories yet intensely driven by FOMO (the fear of missing out ), I made this app on Windows. The major inspiration from the early concept of Mac app Rewind and Black Mirror S1E3 "The Entire History of You" where characters record everything from their eyes, aiming to rewind or search everything that appeared on your computer screen. It provides a series of additional benefits: a non-alterable personal memory stored locally, immunity to lost memory clues, restoring work when software fails, tracing back to the origins of downloaded data, and even treasure-hunting for overlooked information from the past...(for me personally, it has become a joke callback tool when make fun with my friends.)
The implementation is simple. It uses ffmpeg or instant continuous interval screenshots to record screen into small 15-minute fragment files and index them by Windows local OCR api and image embedding. You can also choose to ignore specified programs or screen ranges. All of these goes into a nice-looking local webui interface to rewind or search(by window titles, text keywords, or descriptions of images). With these data, I've also added several overview features where you can view daily and periodic screen times, circadian summaries, and generate monthly lightbox, etc.
I'm big on privacy, so everything happens should only on your computer – nothing in the cloud. I used python to glue things together, it’s transparent and easy to modify by yourself (which might also make it sluggish in performance). And sure, no subscriptions – your data is yours, for free.
I'm not a professional developer. I developed this tool out of personal interest and desire to use it, so it might not be super polished, and you might find a few hiccups. But after using it for a while, I think it's mature and robust enough. If you're interested in this project, welcome to discuss, bring up any issues, or help contribute it better.😺 (I am not sure if similar app already exists on Windows, I know some on Linux like Memento. If you know, please tell me.
Current Drawbacks:
Its data storage is transparent and unencrypted. You can freely use your own structured local data, but you should also be mindful of its protection;
It doesn't house flashy LLM AI capabilities. I believe at the moment, LLM's functionality is largely limited to judging API calls based on user intent and tweaking search results. It hasn't reached the stage where it can directly understand, analyze and recall all data and act on any user intent.(For these processes, it feels like more manual engineering needs to be covered continuously.) The daily title outline and search capabilities currently provided by windrecorder encompass what's feasible, and I don't see how LLM could be better utilized yet. If you have any implementation ideas, free feel to suggest :p
for more information (written in Chinese):
「开源装置」过目不忘记忆搜索器 - 捕风记录仪 | Windrecorder