I’ve always liked to have all my finances information to make analysis, find patterns, improve where I spend my money and make amazing things with that information that I make and I deserve to have. But, hey, the bank only shows you what he wants, so I decided to take it.
After a few research into the web, requests and some code, I saw what I needed and the hard part was to develop the log in.
First of all, I needed to recognise the Pinpad numbers, I could have go with OpenCV, but I’ve been making the TensorFlow tutorial and I thought that, even being a little bit silly, this could be perfect to practice. This model is simpler than the tutorial one but it taught me a lot.
I’m gonna explain the process without going through every step, just the process behind the library that I’m publishing.
It was easy, all that I needed was to pick ten images numbers from ING and pass them correctly to my trainer.
I downloaded the numbers, gave them names and an output value. My training labels were a csv with the file in the first column and the number in the second like
seven.png, 7 and my tests labels is exactly the same, we will always have the same image for each number so… it will work with a 100% of accuracy because with this model we practically are comparing matrices.
I created the tensors, built the linear classifier, trained the model, tested and saved! (This could be another blogpost if you’re enough interested)
Using the model
After training the model I started to develop the module that uses that model. The most important part here is to transform the image that comes from the request into an understandable thing for our classifier.
Now that we have the recognition of the numbers we can handle the auth in ING.
Researching the requests
Then I started to watch any request that came out of ING. First thing I needed is the pinpad info, so I found this request.
I had to send my login information to his endpoint and the pinpad info came back!
With this information I can make the login! Lets transform those base64 strings into images and send them to the model. The server with my model will send us back the number in the image.
Once we have the decoded pins, we need to send back the position of the pins in the pinpad that correspond with the position received.
If our password is 123456, and we have the pinpad [0, 3, 1, 5, 7, 2, 4, 8, 9, 6], using the above response we should send back [5, 1, 6]. Yep, they start to count at 1, not 0.
Now we will send back the resolved pinpad with a returned cookie.
Yeah! We have the ticket and it is gonna let us have full access to our data so lets make the last request in the authentication process: send the ticket back.
Now, we have a full set of headers ready to make any kind of request.
So it’s the moment to share it with the world and start to have the information that we produce and we deserve.
Join the project to make this happen today!
My idea is to have a smart enough model to know each pinpad/login system of any bank and a library to handle their api’s.
Let’s have an Open Banking Data (OBD)
I’ll explain this with further details in my next story.