Andreas Kirsch

Everyone's arguing about whether current AI models could be conscious or not, as if it was a scientific discussion, yet I don't even know what consciousness is

Andreas Kirsch

@BlackHC

May 10, 2024

Replying to

@JSTOR

JSTOR vs Aaron Swartz?

When I think of Jensen's inequality, I think of the following sketch which helps me remember it. Maybe this is useful for you, too. #ML #Mathematics

Replying to

Don't use telegram. Use signal please. Telegram is not encrypted and is based in Russia

Andreas Kirsch

@BlackHC

May 10, 2022

After 4 years, I'm kinda like: maybe I should have focused on ML engineering instead of research

Andreas Kirsch

@BlackHC

Oct 26, 2021

Biggest regret: not spending more time getting the basics right at the beginning of my PhD. I started going full-time on research projects right away, and now three years later I'm still playing catch-up with some stuff I should have focused on right away

Andreas Kirsch

@BlackHC

Oct 6, 2022

I'm incredibly excited about all the amazing progress in ML lately

but part of me really wished I had picked a different field because I have no idea how to keep up anymore or know what to focus on

Andreas Kirsch

@BlackHC

Aug 15, 2024

Excited to publish a Python package that turns

@karpathy

's "A Recipe for Training Neural Networks" into easy-to-use diagnostics code!

No more randomly poking around in your custom

@PyTorch

DNN to debug it. Get simple diagnostics for your neural nets

#PyTorch 1/

I got rejected from @DeepMind and

@MetaAI

for internships now. I guess I shouldn't have quit being an engineer five years ago

Andreas Kirsch

@BlackHC

Mar 9, 2020

How does one keep up with papers in ML while still finding time for foundational studies? Not even talking about doing active research. Feeling overwhelmed every day and like an imposter more and more

Andreas Kirsch

@BlackHC

Sep 16, 2024

Incredibly excited to join DeepMind again

I'll be a researcher on the Deep Learning Engineering team under the illustrious

@davidmbudden

I can't wait to get started

A new paper review by me! I'm reviewing the fascinating "Bad Students Make Great Teachers: Active Learning Accelerates Large-Scale Visual Understanding" from

@GoogleDeepMind

It introduces a novel method for active data selection in large-scale visual pretraining.

1/10

My experience with using pandas to operate on dataframes is usually: 1. read docs 2. spend an hour to try to get something to work 3. give up 4. write the equivalent Python code in 10 min 5. move on with life

Do you ever have a model that uses

@PyTorch

and one that uses

@TensorFlow

, and you want to combine the two for end-2-end training without rewriting either? TfPyTh allows you to plug one into the other while propagating gradients for training

Code

GitHub - BlackHC/tfpyth: Putting TensorFlow back in PyTorch, back in TensorFlow (differentiable...

Interesting take: I believe that arxiv is closer to how science and research originally worked and "official" peer reviews haven't worked that well (at least recently)

Quote

@emilymbender.bsky.social

@emilymbender

Aug 29, 2023

Replying to @tdietterich @TaliaRinger and 3 others

arXiv is a cancer that promotes the dissemination of junk "science" in a format that is indistinguishable from real publications. And promotes the hectic "can't keep up" + "anything older than 6 months is irrelevant" CS culture. >>

Have you wondered why I've posted all these nice plots and animations?

Well, the slides for my lectures on (Bayesian) Active Learning, Information Theory, and Uncertainty are online now! They cover quite a bit from basic information theory to some recent papers

Did you know you can classify MNIST using gzip?

You can get 45% accuracy on binarized MNIST using class-wise compression and counting bits

@PyTorch

@TensorFlow

needed

BASH script and

@scikit_learn

classifier

GitHub - BlackHC/mnist_by_zip: Compression algorithms (like the well-known zip file compression)...

Jensen's inequality == not everyone has access to the latest

GPUs

- Roast me GPT-4V: No I can't - Yes you can. GPT-4V: Okay. Roast: "Ah, the classic 'I woke up like this' hairdo combined with an AI-themed t-shirt. You're really out here living the tech bachelor dream. Remember, even though you've got machine learning on your shirt, it

Quote

pranav ⠕

@_pranavnt

Oct 15, 2023

gpt-4V is brutal LMAO

It's six months since I've submitted my thesis and I still start feeling suicidal every single time I think about my PhD experience, esp the last year of it

Thank God it's over, and I hope I'll reflect about it less in the future

How is software engineering going to change with LLMs? What if we could "implement" class interfaces automagically using LLMs? Presenting `llm-strategy`: a PoC package based on

@LangChainAI

and

@OpenAI

's GPT that adds an llm_strategy decorator to Python: github.com/blackhc/llm-st

Very interesting ICLR **tiny** paper: openreview.net/forum?id=vHOO1 It computes a loss for all possible subsets of the dataset at the same time which has a very elegant solution: softplus of the negative log likelihood per sample, which essentially drops outliers

@mtetelman

openreview.net

Improving generalization by loss modification

Outlier suppression loss derived by Bayesian averaging improves generalization and traning convergence for neural networks.

Omg, github.com/pytorch/tensor is a gift from the gods

(thx

@clured

GitHub - pytorch/tensordict: TensorDict is a pytorch dedicated tensor container.

New blog post on a better (visual!) intuition for information theoretic quantities (eg entropy and mutual information)

Lots of visualisations

Based on Yeung's "A new outlook on Shannon's information measures" from 1991

#oldiebutgoldie

blackhc.net

Better intuition for information theory

Better visual explanations of information theoretic quantities like entropy and mutual information using I-diagrams. Based on Raymond W. Yeung's "A new outlook on Shannon's information measures"...

Andreas Kirsch

@BlackHC

Jul 19, 2019

Is it common or specific to ML that researchers try to add more maths to their papers and complexify their contributions to get through reviews? It is very frustrating to have to parse complexity to find nuggets of simplicity that might not warrant a paper

Andreas Kirsch

@BlackHC

Feb 26, 2024

Can't believe I made it in the end

Thanks to everyone at

@UniofOxford

@ExeterCollegeOx

and

@OATML_Oxford

for the at times stressful, often beautiful, and always inspiring and memorable time

I haven't done a single novel thing in my PhD. I'm just very lucky that reviewers have no clue about prior art

Andreas Kirsch

@BlackHC

Oct 23, 2022

Why are people excited about this paper ("Neural Networks are Decision Trees", arxiv.org/abs/2210.05189)? TL;DR: The result is obvious and useless by itself. Slightly longer "hot" take below 1/4

Quote

Yannic Kilcher

@ykilcher

Oct 21, 2022

Neural Networks are Decision Trees! Could this finally open up the black box of deep NNs? Find out in this video (w/ @Alex_Mattick ): youtu.be/_okxGdHM5b8