Artificial Intelligence is the future. It’s a bit terrifying and amusing at the same time.

I was able to saw a documentary called “AlphaGo” and it is really mind-boggling for me. AlphaGo by Google is basically a computer program that plays the board game “Go” against human .

Fan Hui, a 2 dan European Go champion was sent to compete against Alpha Go and the latter was able to win all of the 5 matches. I am really perplexed by how powerful the AlphaGo is.

Later on AlphaGo competed with Lee Sedol (ranked 9-dan), one of the best players of Go. Lee Sedol was confident that the AlphaGo will not beat him because he observed the matches of Fan Hui and the AI, and he thought that the AlphaGo was not quite as the same level as him yet. Shockingly, even the world’s best Go player was defeated in 4 out of 5 games played. (Still Kudos to Lee Sedol for being the only human to beat AlphaGo out of all of the official matches conducted.

It’s really flabbergasting how a human trained artificial intelligence beat the very same humans that created it. It is good to think about, and the applications of AI is beyond our imagination, but it is also scary to think that an Artificial Intelligence was able to beat human intuition so easily.

AI is a great step towards progress but we have to be more mindful so that no problems arises from it.

In China, they are very serious in making themselves the country leading in terms of AI advancement. They have the data and the man power. If they were able to succeed with their mission, it would definitely make major changes in the sociopolitical and economical landscape of the world.

That’s it. That’s the evidence. I am a fashion enthusiast.

A lot of people tell me “Queenie, you have way too many clothes.” But I always shrug and think that I always don’t have enough clothes, people are subjective anyway. I find myself almost everyday thinking “Gosh. I don’t have enough clothes.”

But when I learned about machine learning, the reality struck hard. I do spend a lot of time looking for clothes to buy. I am too enthusiastic with clothes.

My shoppee daily discover almost always recommends clothes for me which often leads me to making an impulse purchase, and as you can see on the screenshot, I am also looking for a clothes storage as well. Why? Because I simply have too much clothes :(

Personally, I don’t like seeing recommendations like this because I kept on spending on clothes when I clearly have way too many of them. But that’s just how shoppee works and it is their strategy to acquire more transactions. I might just as well uninstall it.

To me, personally, statistics is just another intimidating mathematical science subject, but we see it almost everyday and can make use of it in the aspect of our day to day lives.

As an aspiring data scientist, I have no choice but to embrace statistics, as it is somehow intertwined with my field of interest. A data scientist would definitely be working with data, probably like a unicorn who will see through data collection until the end, or maybe just provide insights and analysis with a clean data set. And what would a data scientist do with these data? a data scientist would have to assess which data, and specific variables to be used; analyze the data; identify patterns and trends; applying models. In essence, data scientists utilize statistics in a lot of step of their work. In data analysis, they might use simple mean, median, mode, or maybe resort to more complicated statistical tools such as test of correlations. They are using statistics in order to generate insights with the data that they have. It is really necessary to have sufficient knowledge in statistics, to become a data scientist.

There are a lot of statistical techniques that will help a data scientist. For an instance, regression (linear) is very useful to predict next sets of data, hence, it can help a data scientist form recommendations. Sampling methods are basic, but very much needed as well. A data scientist might one day encounter a very huge data and if the data scientist do not have direct access to the data, the time for data extraction and transfer would be a loss, hence an adequate sampling is the next best thing.

Let’s say you are given multiple data sets with over a hundred thousand rows and ten columns each. What do you do?

Of course the wise answer would be to explore and clean the data (other than praying that your laptop could handle the load). Exploring the data will be identifying which data do you need and cleaning the data would be removing those data that are not remotely relevant to the study, it is an option to remove the null values as well.

If you do not explore and clean the data, you will not be able to see a greater picture because the other data that you don’t need acts as noise. You can spend time coding and making it work for this variable, but it turns out you have a better option that you would have known if you made time to explore the data. Evidently the process of exploring and cleaning the data takes a lot of time (Our mentor even said that this process takes weeks, only to do the data science thing for two hours), but it is a process you can not skip. If you do, you may spend days and days trying to process a very heavy file (longer time to load), only to find it futile because the data is dirty and some of the variables are not standard with each other. Moreover, having large amount of data, especially if its personal, poses a security risk, so it is better to explore and clean the data before you put those data science magic.

I was able to acquire an anonymous data about insurance and I would like to gather some insights about it. I will try to make use of pandas and matplotlib libraries . Let’s have a look with the data first.

Okay, so there are few null values. I would like to disregard them so that they will not have much bearing when I employ statistics.

All good, we were only able to remove 4 rows. I also noticed that the label of gender is inconsistent some are labeled male, female, and there are also woman and man, so I have to sync them by running these code:

df = df.replace({“man”:”male”})
df = df.replace({‘woman’:’female’})

Same goes for the region.

What I want to know is which group pays the most insurance. I would like to know it according to the following categories: bmi, and number of children.

code: print(df[[‘sex’, ‘charges’]].groupby([‘sex’]).agg([“mean”, “max”, “min”]))

As you can see, the male population is pays higher insurance. It is possible to consider that it is because the average number of children and bmi of men are slightly higher as well.

For the number of children, what is very noticeable is that those with the most number of children pay the least amount for insurance. Whereas the other age group does not have a wide range of the amount they are paying for.

Meanwhile, let’s look closer at the BMI. I was able to create a new column for my data which is the BMI status. It tells us whether the person is underweight, normal, overweight or obese.

I used the below code to create a new column:

rating = [
(df[‘bmi’]< 18.5),
(df[‘bmi’]>= 18.5) & (df[‘bmi’] < 24.9),
(df[‘bmi’]>= 24.9) & (df[‘bmi’] < 29.9),
(df[‘bmi’]>= 29.9),
]

label = [‘underweight’,’normal’,’overweight’,’obese’]
df[‘bmi_stats’] = np.select(rating,label)

There are also more male that has a BMI status of overweight to obese. As you can see as well, obese people pay most for the insurance.

Let’s try to visualize the data.

In here we can see that the average insurance being charged to men is a bit higher than with what is being charged with women.

We would like to know first if the number of children may have affected the difference. keep in mind that the average number of children for male is 1.12, whereas 1.07 for female. Looking at the chart below, we can see that the average charges being paid according to the number of children is quite close per category, with the exception of those who have 5 children. Hence, what we can only infer here is that it reaches a point that the more children you have, the less willing you are to pay for high insurance.

Meanwhile, the graph below shows that more obese and overweight people are paying for higher insurance.

And since we can see that there are more overweight and obese male than female, we might consider the BMI status as a factor for the price difference between insurance.

Feel free to look at my codes here: https://github.com/ftwqueenie/pythoncharts

For this week’s task, we were asked to recalibrate our quiz that we made the Saturday before. What is expected to this new version of quiz is to include CRUD techniques (create, read, update, delete).

As a refresher, let me show you the original quiz I made:

So as you can see at the top part, I declared the variables that I will be using throughout the code. Afterwards, I declared the functions.

My functions are:

• intro(name) — this is called at the start of the program wherein a user is asked to input his/her name.
• rating(score) — this function will compute the ratings of the users. It will assess if the user needs to study more or it already passed.
• quiz(question,answers) — this is the main part where the questions are being loaded and the user input answers are being compared to the answer key. I used a separate list for questions and answers and just utilized an iterator to match the indices of respective questions and answers. A better way to do it would be through dictionary.

Here comes the modifying part.

I decided to add another function called loadscores. In this function, my goal is to write the names, scores, and the ratings of the user (recently added to say if a user failed or passed) in a text file.

I made 5 version of the quizzes and opened more than 10 tabs of google chrome just to arrive at this:

since I used a list as a place holder for the names, scores, and ratings of the users, I can use the for loop to iterate through the user details.

At the beginning of the code I imported from os import path so that I can have a way to check if the text file exists. This is important because this will determine if i should write the text file or append it (thank you google!)

After determining whether I should append the text file or create a new one, I coded the lines for the content of every column of the text file (name, score, rating).

Let me modify first the code so that the square brackets wont be included when writing using .replace()

Let’s check out the file:

Works well!

Next is the tricky part. Deleting the first entry whenever the scoreboard reaches 10 rows. For a simpler testing I just used 3 as the limit so that it will be faster to run the test.

So in here I created a variable called last_list taht reads the existing scoreboard.txt. The for loop will count the number of lines in the file and placed to a variable called numscores.

The if loop is the condition that sets when to delete an item. So as I’ve said I will set it to 3, and if it reaches 3, the first row in the text file will be deleted.

I created a variable called newscore as a holder for rewriting the new lines in the file. Prior to deleting, there are 4 lines already in the txt, and now since we deleted the first index, we are expecting t write 3 lines in our text file.

Let’s demonstrate if its correct

Remember that on our first run we already have 1 item in the text file. the record of user Queenie. So we just have to run 2 more times.

Now user Dylan and Kier were able to take the quiz. After running the code again for another user, we are expecting the record for Queenie to be deleted.

Our del works! :D

Extra: Converting it to a csv file.

If we want to study the data that was stored into the text file, It is best to have it in a CSV form. So I added extra codes (in a separate py) to convert my txt file into a csv with 3 columns.

Just a few lines of code but very powerful :)

Here’s the result:

Check out my code at git hub :)

Here’s the link:

ftwqueenie/quizwithCRUD

Hello again! Apparently I misunderstood the instruction so I need to have modifications.

So task 1 should be able to display the 10 most recent game plays in the terminal, rather than writing a file for it.

It is pretty much the same logic with writing it in file, but this time I just have to print the scores. I still kept the write to file function though.

The second task is to be able to apply CRUD on the questions (at first I applied it on the scores part).

For that, I made an admin mode where I can have the options to delete and add questions.

I just patterned my codes with the previously made function where I write the scores in the text file and access the lists I made.

I have reuploaded it under the same repo but do checkout https://github.com/ftwqueenie/quizwithCRUD.

Let’s Keep a Record of Your Score was originally published in The Startup on Medium, where people are continuing the conversation by highlighting and responding to this story.

Notebooks are very cool. They were very effective to use when teaching. Our IM, Sir Myk, and Junior DS, Shane, both used notebook when they were teaching us.

What I like with notebooks is that you can have a portion for texts including formats such as heading, subtitle, bold face, italicized and more. It will not interrupt at our code at all.

As you can see, using notebook, I was able to organize my code. So that it will be easier to understand. I have a title, and labels for each section of the code. I was able to compartmentalize my codes so that I can easily find errors when I try to run it, however, the down side of doing that is that I have to run the other cells as well. Nevertheless, it is a neat way to present my codes.

Let’s test my quiz!

It works well! I even got a perfect. As my app told me, I’m a walking atlas!

Now when I presented my code, I was told that I could make use of the functions in order to make my code better. This time I will be using Spyder, an IDE, to recreate my codes with the suggested modifications.

I was able to modify the code at Spyder. In Spyder, I have no option to create a text field with customizable fonts and styles, however I tried to make do using comments. For multiple line comments, I just have to enclose it with three quotation marks on both ends, while on single line comment, I just have to put a hash prior to the comment.

What I really like about using Spyder is that it the color varies for variables. It makes it really easy to identify.

I also really liked the variable explorer part, wherein I can see the summary of the variables that I declared. I especially loved it because it showed me the placeholder for my score, which is a list called ‘correct’. I have never printed it anywhere in my code, so it’s good to see that it is working well on the background.

Like colab, it has a built-in interface where you can run the codes (not sure if all notebooks do that), but as I tell you, this is very convenient as compared to having to run the code in cmd or terminal.

Overall, I think both tools are great, but if you really have to ask me, I prefer the Spyder because I can explore the variables. I think it is a really helpful function.

Bonus: i have uploaded my notebook and my work in spyder at my git hub

Check out this link: https://github.com/ftwqueenie/the-capital-quiz

I am now following this link. https://docs.google.com/document/d/1Zxm03eDfWy8gDdyFtYRDr2Hf01rSLmKRz2Qv30vrMxU/edit#heading=h.dyvdx1vxlkac

Here is my github account :) link: https://github.com/ftwqueenie

I already tried making a repository before and cloned it to my laptop, but I already forgotten how. So I have to make an ew one and I called it git-and-colab-basics.

I was able to successfully clone the repository in my laptop and followed the instructions.

I copied my revised work of the activity last Saturday (filename: 200917_QDBP.ipnynb) so that I could add it in my repository.

I had a major problem with commit part as it kept on asking for my credentials and I got confused at first. It seems like my cmd doesn’t know my git hub acct yet. but with a little help, I was able to overcome it and successfully pushed the new file I added to the repository I made.

This is my “rush commit”

I also cloned a data scientist cheat sheet in my laptop. Once I have time, i’ll be sure to check them out.

For Jupyter, it looked quite similar with the interface of colab. I just made a new notebook called Git Basics. I just redid the sample that we did last week and it worked the same even when I’m using a different tool.

So what I did is just to import GrabDataSeer.csv file in the same directory I’m currently working in and imported it in my code. I removed all the rows that contains null values (unallocated) and saved the new file as gitbasics_removedmissing.csv

So now I have the notebook, the original csv and the reworked csv in my directory.

Finally I recreated the steps using SPYDER. See my code below :)

Running it didn’t print anything.

But when I checked the directory where I saved the file, there it is. i was a ble to modify the GrabDataSeer.csv once again.

And that’s it for now. I’m kind of getting the gist of the basics. I am a bit anxious for what’s in store for me next class!

As I take this journey towards the much yearned Data Science career, I am tasked to retrace my steps and relearn python coding. I’m ecstatic to find out that while I can not recreate the codes on my own, I can actually understand them if I read them. The python syntax is still flickering a bit in my mind.

Following the video I was watching, I saw how to declare variables and how to import items in your code (key: pandas!!!) So where do I start my coding? of course, to print Hello World!

So moving on, I also declared variables and had them print

But I remember, back then I was able to ask inputs from users whenever I’m adding something. I want to try that again but my brain failed me. Luckily, my trusty friend stack over flow came through, although it is definitely not the function I’m expecting and I’m hoping I could remember how i did it before, but nevertheless i got to ask user inputs to have a sum! (however I can’t concatenate the text and the variable sum :( )

I really do have to take a crash course to Python. I can’t make my former knowledge work anymore. Let’s check if I still remember the command prompt. This is where we load our codes back then

So the content of the video is quite unfamiliar with me us we have not maneuvered cmd before (except cd). So in the tutorial vids, I learned several things:

1. DIR — lists all the items in the directory

You can also use DIR without changing the current directory

2. rmdir and mkdir — removes and makes directories

rmdir /s — removes everything in the directory, including all of the directories it contains

3. Tree — displays everything inside the directory including the directories inside of those directories and so on, in a tree form. This one is pretty cool and I found out that I have a lot of items in my C drive and it did not stop for 3 minutes until I hit CTRL C.

4. Color — my favorite! I get to change colors in cmd and I am fond of the matrix team as it feels like I am a master hacker.

5. Attrib — shows the attributes in the directory. You can also change the attributes of files.

6. Del — deletes files

7. Echo > — you can use echo to create a file, with text that was indicated in the echo. Double greater than (>) sign will not overwrite the file and just append the additional text you input.

8. Copy — will allow you to copy files or folders into a specified destination, and xcopy will allow you to copy everything including directories

9. Move — it enables us to move directories or files from source to a destination

10. Rename — will allow you to rename files

So that’s it for now. I’m looking forward to learn more as I take the Data Science journey with For The Women foundation.