AI hulpeloos tegenover Black Swans

Ik herlees Nicholas Nassim Taleb’s Fooled By Randomness. Heel veel voorspellingskracht wordt gevonden in historische data. Maar niet de Black Swans. Ik moet denk aan de hype rond AI die momenteel heerst.

AI en ML leven van historische data. Black Swan events komen niet voor in historische data.

De AI is dus net zo stom als mensen: onze AI ziet grote hoeveelheden witte zwanen. Als je de AI vraag welke kleur een zwaan heeft, zegt hij dus: wit. Zwarte zwanen bestaan ook voor de AI niet.

AI is waardeloos waar het gaat om Black Swan Events.

ChatGPT zegt het zelf:

AI systems, including those based on advanced algorithms like GPT-3.5, have limitations when it comes to predicting black swan events. Black swan events are extremely rare, unpredictable, and have a significant impact. These events are, by definition, unforeseen and often fall outside the range of normal expectations.

AI models, including machine learning algorithms, are generally trained on historical data. If black swan events have never occurred in the training set, the AI model would not have learned to predict them explicitly. Traditional AI models work based on patterns and correlations found in the data they were trained on. They are not capable of inventing entirely new and unprecedented events beyond the scope of their training data.

However, it’s essential to note that AI can be helpful in preparing for unexpected events, even if they cannot predict black swan events directly. Here’s how:

Despite these capabilities, it’s important to maintain a cautious approach when relying on AI for preparing for black swan events. Human judgment, expert analysis, and continuous monitoring are essential elements in mitigating the impacts of truly unforeseen and exceptional events.

AI, negeer alsjeblieft de wereld die voor je ligt

Malcolm Gladwell legt in zijn nieuwe boek Talking to Strangers uit dat we niet op intuïtie moeten vertrouwen. In plaats daarvan moeten we ons realiseren dat we omgaan met mensen van wie de aannames, perspectieven en achtergronden verschillen van de onze, en voordat we conclusies trekken eerst intern onderzoek doen.
Dit lijkt aan te sluiten bij het onderzoek van Daniel Kahneman, zoals gepubliceerd in Thinking Fast and Slow, waarin ons Systeem 1-denken ons tot snelle conclusies leidt die ons Systeem 2-denken zal proberen te rechtvaardigen.

Nu de link naar AI:
Kunstmatige Intelligentie neemt mogelijk niet alleen beslissingen omdat het meer (dieper) weet over een onderwerp en al die informatie snel kan verwerken. Het lijkt er ook op dat AI betere beslissingen kan nemen omdat het veel informatie kan negeren die in het hoofd van mensen zit, en het kan de fysieke aanwezigheid van de dingen voor ons negeren.
Als dat zo is, kan AI maar beter niet menselijker worden. Dat zou rampzalig zijn voor de kwaliteit van haar beslissingen.

Ubuntu

Deze zaterdag installeerde in Ubuntu Linux op de oude laptop van mijn moeder, een Dell Vostro Intel Core i3 met 4GB uit ik weet niet welk jaar, ik denk 2013. De installatie was een makkie, met balenaEtcher een image naar een USB stick geschreven.

Nu nog kijken wat ik er mee ga doen. Een website hosten die ik helemaal zelf programmeer in Django/Python?

Ubuntu betekent zoveel als openstaan voor anderen, je wijden aan anderen, compassie en respect tonen.

Wie staat open voor mijn Ubuntu laptop ?

Programming languages and what’s next

My review of programming languages I learned in during my years in IT.

BASIC

On the TI99-4a.
Could do everything with it. Especially in combination with PEEK and POKE. Nice for building small games.
Impossible to maintain.
GOTO is unavoidable.

Assembler

In various variants.
Z80, 6802, PDP 11, System 390.
Fast, furious, unreadable, unmaintainable.

Algol 68

Liked this language. REF!
Have only seen it run on DEC 10. Mainly used in academic environments (in the Netherlands?)?

Pascal

Well. Structured. Pretty popular in the early 90s.
Again is this widely adopted?

COBOL

Old. Never programmed extensively in it – just for year 2000.
Totally Readable.
Funny (rediculous) numbering scheme.
Seems to be necessary to use GOTO in some cases which I do not believe.

Smalltalk

Beautiful language.
Should have become the de facto OO programming language but failed for unclear reasons.
Probably because it was way ahead of it’s time with it’s OO base.

Java

Totally nitty gritty programming language.
Productivity based on frameworks, which no one knows which to use.
Never understood why this language was so widely adopted – besides it’s openness and platform independency.
Should never have become the de facto OO programming language but did so because Sun made it open (good move).
Far too many framework needed. J(2)EE add more complexity than it resolves.
Always upgrade issues. (Proud programmer: We run Java! Fed up business person: Which Java?)

Rexx

Can do everything quickly.
But nothing structurally.
Ugly code. Readable but ugly.
Some very very strong concepts.

Php

Hodge-podgy language of programming and html.
Likely high programmer productivity if you maintain a stark discipline of programming standards. Stark danger of creating unmaintainable crap code mix of html and php.

Python

Nice structured language.
Difficult to set up and reuse.
Can be productive if nitty gritty setup issues can be overcome.

Ruby (on Rails or off-track)

Nice, probably the most elegant OO language. Too nitty gritty to my taste still. Like it though.
I would start with this language if I had to start today.

What is next?

Visual programming? Clicking building blocks together?

In programming we should maybe separate the construction of applications from the coding of functions (or objects, or whatever you call the lower level blocks of code.

Programming complex algorithms (efficiently) will probably always remain a craft for specialists.

Constructing applications from the pieces should be brought to a higher level.

The industry (well – the software selling industry) is looking at microservices but that gives operational issues and becomes too distributed. We need a way to build a house from software bricks and doors and windows and roof elements.

Probably we need more standards for that.

Some bold statements

AI systems “programming” themselves is nonsense (I have not seen a shred of evidence).
AI systems are stochastical systems.
Programming is imperical.

In summary, up to today you can not build software without getting into the nitty gritty very quickly.
It’s like building a house but having find your own tree and rocks first to cut wood and blicks from.
And then construct nails and screws.
A better approach to that would help.

What do you think is the programming language of the future? What need should it address.

In Layman’s Terms: (Visa) Format-Preserving Encryption

One of my clients asked me about Visa Format-Preserving Encryption. Could look into this. I am sharing a summary here.

FPE is …

Format-Preserving Encryption (FPE) is about encrypting so-called structured data, such as credit card or Social Security numbers.

With FPE you can encrypt data in such a way that does it not alter the data format.

Your data may be credit card numbers, account numbers, social insurance numbers, addresses, postal code, FPE will encrypt the data and the outcome will look like a credit card number, account number, social insurance number, address, postal code.

That’s it.

With this you can encrypt data per field without having to make big modifications to your existing application.

Then we come to the real goal.
We can have better data privacy and security. We can agree and adhere to certain industry standards for data privacy and security, such as PCI DSS in the Payment Card industry.

And the Visa?

Visa have invented and patented an algorithm to realize FPE and use this in their card handling. Called VISA Format-Preserving Encryption.visa format-preserving encryption

Why do we need this format preservation?

Sensitive data must be encrypted so that malicious people can not read that data. In financial transactions this is of course very important.

A problem is that with encryption the format of field could change. Traditional encryption algorithms take data as a byte stream, and turn it into another byte stream.

If you have a string of text, such as a name, after encryption this encrypted field may become larger than the original field, it may contain numbers or funny characters, or it may have become be a bigger field. Or if you have an account number, after encryption there may be characters in the result. Or there may be more digits in the result than in the account number.

This change of format of fields can cause several issues in applications.

Column sizes in databases may be too small to contain encryption results.
Fields in programs may become too small
Test data for application can become problematic.
It can become impossible to implement if these fields are used as keys in relational databases.
Your analytics may become problematic.

To prevent all these issues Format Preserving Encryption was invented. With it we can encrypt fields, but process these field in the programs as if they were not encrypted. The program is not aware of the encryption.

Clearly I am leaving out details. Fill me in where you feel necessary, I’s appreciate it.