And even then I don’t think many of them truly understand what’s going on in there.
That’s just the thing about neural networks: Nobody actually understands what’s going on there. We’ve put an abstraction layer over how we do things that we know we will never be able to pierce.
I have a masters degree in statistics. This comment reminded me of a fellow statistics grad student that could not explain what a p-value was. I have no idea how he qualified for a graduate level statistics program without knowing what a p-value was, but he was there. I’m not saying I’m God’s gift to statistics, but a p-value is a pretty basic concept in statistics.
Next semester, he was gone. Transferred to another school and changed to major in Artificial Intelligence.
I have a bachelor’s and master’s in computer science, specialised in data manipulation and ML.
The problem with AI is that you don’t really need to understand the math behind it to work with it, even with training. Who cares how the distribution of the net affects results and information retention? who cares how stochastic gradient descent really works? You get a network crafted by professionals that gets X input parameters, which modify the network’s capacity in a way that’s given to you, explained, and you just press play in the script that trains stuff.
It’s the fact that you only need to care about input data quality and quantity and some input parameters that freaking anyone can work with AI.
All the thinking on the NN is given to you, all the tools to work with training the NN are given to you.
I even worked with darknet and Yolo and did my due diligence to learn Yolov4, how it condensed info and all that, but I really didn’t need to for the given use case. Most of the work was labelling private data and cleaning it thoroughly. Then, playing with some Params to see how the final results worked, how the model over fitted…
That’s the issue with people building AI models, their work is more technical that that of “prompt engineers” (😫), but not much.
When you’re working at the algorithm level, you get funny looks… Even if it gets to state of the art results, who cares because you can throw more electricity and data at it instead.
I worked specifically on low data algorithms, so my work was particularly frowned upon by modern ai scientists.
I’m not doxxing myself, but unpublished work of mine got published in parallel as Prototypical Networks in 2017. And everyone laughed (<- exaggeration) at me researching RBFs which were considered defunct. (I still think they’re an untapped optimization.)
It all became basically magic, blind trial and error roughly ten years ago, with AlexNet.
After AlexNet, everything became increasingly more and more black box and opaque to even the actual PhD level people crafting and testing these things.
Since then, it has basically been ‘throw all existing information of any kind at the model’ to train it better, and then a bunch of basically slapdash optimization attempts which work for largely ‘i dont know’ reasons.
Meanwhile, we could be pouring even 1% of the money going toward LLMs snd convolutional network derived models… into other paradigms, such as maybe trying to actually emulate real brains and real neuronal networks… but nope, everyone is piling into basically one approach.
Thats not to say research on other paradigms is nonexistent, but it is barely existant in comparison.
Way back in the 90s when Neural Networks were at their very beginning and starting to be used in things like postal code recognition for automated mail sorting, it was already the case that the experts did not know why it worked, including why certain topologies worked better than others at certain things, and we’re talking about networks with less than a thousand neurons.
No wonder that “add shit and see what happens” is still the way the area “advances”.
Il’ll give you the point regarding LLMs… but conventional neural networks? Nah. They’ve been used for a reason, and generally been very successful where other methods have failed. And there very much are investments into stuff with real brains or analog brain-like structures… it’s just that it’s far more difficult, especially as have very little idea on how real brains work.
A big issue regarding digitally emulating real brain structures is that it’s very computationally expensive. Real brains work using chemistry, after all. Not something that’s easy to simulate. Though there is research in this are, but that research is mostly to understand brains more, not for any practical purpose, from what I know. But also, this won’t solve the black box problem.
Neural networks are great at what they do, being a sort of universal statistics optimization process (to a degree, no free lunch etc.). They solved problems that failed to be solved before, that now are considered mundane. Like, would anyone really think it would be possible to have your phone be able to detect what it was you took a picture of 15 years ago? That was considered to be practically impossible. Take this xkcd from a decade ago, for example https://xkcd.com/1425/
In addition, there are avenues that are being explored such as “Explainable AI” and so on. The field is more varied and interesting than most people realize. And, yes, genuinely useful. And not every neural network is a massive large scale one, many are small-scale and specialized.
This method is definitely a great way to achieve some degree of explainability for images, but it is based on the assumption that nearby pixels will have correllated meanings. When AI is making connections between far-away features, or worse, in a feature space that cannot be readily visualized like images can, it can be very hard to decouple the nonlinear outputs into singular linear features. While AI explainability has come a long way in the last few years, the decision-making processes of AI are so different from human thought that even when it can “show its work” by showing which neurons contributed to the final result, it doesn’t necessarily make any intuitive sense to us.
For example, an image-identification AI might identify subtle lens blur data to determine the brand of camera that took a photograph, and then use that data to make an educated guess about which country the image was taken in. It’s a valid path of reasoning. But it would take a lot of effort for a human analyst to notice that the AI is using this process to slightly improve its chances of getting the image identification correct, and there are millions of such derived features that combine in unexpected ways, some logical and some irrationally overfitting to the training data.
That’s just the thing about neural networks: Nobody actually understands what’s going on there. We’ve put an abstraction layer over how we do things that we know we will never be able to pierce.
deleted by creator
I have a masters degree in statistics. This comment reminded me of a fellow statistics grad student that could not explain what a p-value was. I have no idea how he qualified for a graduate level statistics program without knowing what a p-value was, but he was there. I’m not saying I’m God’s gift to statistics, but a p-value is a pretty basic concept in statistics.
Next semester, he was gone. Transferred to another school and changed to major in Artificial Intelligence.
I wonder how he’s doing…
I have a bachelor’s and master’s in computer science, specialised in data manipulation and ML.
The problem with AI is that you don’t really need to understand the math behind it to work with it, even with training. Who cares how the distribution of the net affects results and information retention? who cares how stochastic gradient descent really works? You get a network crafted by professionals that gets X input parameters, which modify the network’s capacity in a way that’s given to you, explained, and you just press play in the script that trains stuff.
It’s the fact that you only need to care about input data quality and quantity and some input parameters that freaking anyone can work with AI.
All the thinking on the NN is given to you, all the tools to work with training the NN are given to you.
I even worked with darknet and Yolo and did my due diligence to learn Yolov4, how it condensed info and all that, but I really didn’t need to for the given use case. Most of the work was labelling private data and cleaning it thoroughly. Then, playing with some Params to see how the final results worked, how the model over fitted…
That’s the issue with people building AI models, their work is more technical that that of “prompt engineers” (😫), but not much.
When you’re working at the algorithm level, you get funny looks… Even if it gets to state of the art results, who cares because you can throw more electricity and data at it instead.
I worked specifically on low data algorithms, so my work was particularly frowned upon by modern ai scientists.
I’m not doxxing myself, but unpublished work of mine got published in parallel as Prototypical Networks in 2017. And everyone laughed (<- exaggeration) at me researching RBFs which were considered defunct. (I still think they’re an untapped optimization.)
I’d argue we know exactly what’s going on in there, we just don’t necessarily, know for any particular model why it’s going on in there.
But, more importantly, who is going on in there?
And how is it going in there?
Not bad. How’s it going with you?
That’s what we’re trying to find out! We’re trying to find out who killed him, and where, and with what!
The real question is where it’s going on?
Excellent opportunity for a “that’s what she said” joke.
Ding ding ding.
It all became basically magic, blind trial and error roughly ten years ago, with AlexNet.
After AlexNet, everything became increasingly more and more black box and opaque to even the actual PhD level people crafting and testing these things.
Since then, it has basically been ‘throw all existing information of any kind at the model’ to train it better, and then a bunch of basically slapdash optimization attempts which work for largely ‘i dont know’ reasons.
Meanwhile, we could be pouring even 1% of the money going toward LLMs snd convolutional network derived models… into other paradigms, such as maybe trying to actually emulate real brains and real neuronal networks… but nope, everyone is piling into basically one approach.
Thats not to say research on other paradigms is nonexistent, but it is barely existant in comparison.
Way back in the 90s when Neural Networks were at their very beginning and starting to be used in things like postal code recognition for automated mail sorting, it was already the case that the experts did not know why it worked, including why certain topologies worked better than others at certain things, and we’re talking about networks with less than a thousand neurons.
No wonder that “add shit and see what happens” is still the way the area “advances”.
Il’ll give you the point regarding LLMs… but conventional neural networks? Nah. They’ve been used for a reason, and generally been very successful where other methods have failed. And there very much are investments into stuff with real brains or analog brain-like structures… it’s just that it’s far more difficult, especially as have very little idea on how real brains work.
A big issue regarding digitally emulating real brain structures is that it’s very computationally expensive. Real brains work using chemistry, after all. Not something that’s easy to simulate. Though there is research in this are, but that research is mostly to understand brains more, not for any practical purpose, from what I know. But also, this won’t solve the black box problem.
Neural networks are great at what they do, being a sort of universal statistics optimization process (to a degree, no free lunch etc.). They solved problems that failed to be solved before, that now are considered mundane. Like, would anyone really think it would be possible to have your phone be able to detect what it was you took a picture of 15 years ago? That was considered to be practically impossible. Take this xkcd from a decade ago, for example https://xkcd.com/1425/
In addition, there are avenues that are being explored such as “Explainable AI” and so on. The field is more varied and interesting than most people realize. And, yes, genuinely useful. And not every neural network is a massive large scale one, many are small-scale and specialized.
I take your critiques in stride, yes, you are more correct than I am, I was a bit sloppy.
Corrections appreciated =D
Hopefully I don’t appear as too much of a know-it-all 😭 I often end up rambling too much lmao
It’s just always fun to talk about one’s field ^^ or stuff adjacent to it
Oh no no no, being an actual subject matter expert or at least having more precise and detailed knowledge and or explanations is always welcome imo.
You’re talking to an(other?) autist who loves data dumping walls of text about things they actually know something about, lol.
Really, I appreciate constructive critiques or corrections.
How else would one learn things?
Keep oneself in check?
Today you have helped me verify that at least some amount of metacognition is still working inside of this particular blob of wetware, hahaja!
EDIT:
One motto I actually do try to live by, from the Matrix:
Temet Nosce.
Know Thyself.
… and a large part of that is knowing ‘that I know nothing’.
Feature Visualization How neural networks build up their understanding of images
https://distill.pub/2017/feature-visualization/
This method is definitely a great way to achieve some degree of explainability for images, but it is based on the assumption that nearby pixels will have correllated meanings. When AI is making connections between far-away features, or worse, in a feature space that cannot be readily visualized like images can, it can be very hard to decouple the nonlinear outputs into singular linear features. While AI explainability has come a long way in the last few years, the decision-making processes of AI are so different from human thought that even when it can “show its work” by showing which neurons contributed to the final result, it doesn’t necessarily make any intuitive sense to us.
For example, an image-identification AI might identify subtle lens blur data to determine the brand of camera that took a photograph, and then use that data to make an educated guess about which country the image was taken in. It’s a valid path of reasoning. But it would take a lot of effort for a human analyst to notice that the AI is using this process to slightly improve its chances of getting the image identification correct, and there are millions of such derived features that combine in unexpected ways, some logical and some irrationally overfitting to the training data.