This blog will give you the info you need to make an informed decision about which popular GenAI tool to use and what to be aware of regarding data and privacy.

TLDR? Go to the end of the article for tool recommendations

Should you be concerned about the data you enter in to Gen AI tools?

A general rule of thumb is, “If it’s free, you are the product”. With Gen AI, the business attracts more users by offering a service for free and then gathers the data they provide to be put to use in the way they want. While free stuff is great today, this maybe not necessarily benefit you in the long run.

The technology which powers ChatGPT and similar products, Large Language Models (LLMs), use training data to infer patterns which could later replicate and expose information at a different point in time. While attempts are made to cleanse the model, there’s no memory as such in an LLM, so deleting data doesn’t work as you’d assume it would with a regular file on your hard drive.

“When you enter data into a large language model (LLM), the model turns your input into numbers and patterns that represent meaning, not the original words or data. After this transformation, it’s very hard to pick out or separate the exact pieces of information you put in, because everything gets mixed together as part of the model’s understanding“ — how LLMs work explained by perplexity.ai

You should be aware of what the different Gen AI chatbots and assistants do with user inputs (text, images, documents etc) and be cautious about the sensitivity of the data you enter, especially if it’s about other people or for work.

Enterprise vs Consumer models

Nearly all providers make this key distinction.

If your employer is providing you with a chatbot service, odds are you’re on an enterprise model. Still, you should check if there is a policy or guidance on how you use the tool at work responsibly.

If you’ve signed up yourself, rather than signing in via your organisation, you’re likely using a consumer plan. These can be free or paid.

Below I will outline popular providers and what they do with your data, based on the plan (this is for Europeans — US data regulations are likely less strict).

ChatGPT (by OpenAI)

What does it do with you data?

Any individuals data given to ChatGPT could be used to train it’s models unless you opt out (in settings) or use a ‘temporary’ chat.

ChatGPT uses both automated and manual, human processing methods (but less often, only when required e.g. for safety training).

How long does it retain your data for?

This depends on your data settings, but conversations will be retained for a minimum of 30 days.

Uses: Chat, image generation, video generation, coding.

Plans: ‘Free’ ($0/£0 p/m), ‘Pro’ ($20/£15 p/m), ‘Plus’ ($200/£150 p/m)

Try ChatGPT here (no sign-in needed):

https://chatgpt.com/

Read more:

• EU Privacy Policy https://openai.com/en-GB/policies/eu-privacy-policy/
• Consumer Privacy https://openai.com/consumer-privacy/
• Plans and Pricing https://openai.com/chatgpt/pricing/
• Data Privacy FAQs https://help.openai.com/en/articles/8554402-gpts-data-privacy-faqs
• Data Controls https://help.openai.com/en/articles/7730893-data-controls-faq
• How To Delete Your Account https://help.openai.com/en/articles/6378407-how-to-delete-your-account
• How ChatGPT and our foundation models are developed https://help.openai.com/en/articles/7842364-how-chatgpt-and-our-foundation-models-are-developed

Claude (by Anthropic)

What does it do with you data?

If you’re using it, Claude won’t use your data to train it’s models, no matter which plan you use, UNLESS one of three things happen:

1. You give feedback via a thumbs up or down rating
2. You reach out to the service provider with a request
3. You input content which needs a trust or safety review

Your location is inferred from your IP address.

How long does it retain your data for?

Text is retained for 2 years. Audio recordings are deleted as soon as transcribed to text.

Uses: Chat, analyse but not generate images, coding (Claude Code is only available via the ‘Max’ plan)

Plans: Free ($0/£0 p/m), Pro ($20/£15 p/m), Max ($200/$150 p/m)

Try Claude here (Sign-in needed):

https://claude.ai

Read more:

• https://privacy.anthropic.com/en/
• https://privacy.anthropic.com/en/articles/10023548-how-long-do-you-store-my-data
• https://privacy.anthropic.com/en/articles/10067979-what-personal-data-is-collected-when-using-dictation-on-the-claude-mobile-apps
• https://privacy.anthropic.com/en/articles/11186740-does-claude-use-my-location
• https://www.anthropic.com/pricing
• https://support.anthropic.com/en/articles/8325621-i-would-like-to-input-sensitive-data-into-free-claude-ai-or-my-pro-max-accout-who-can-view-my-conversations
• https://support.anthropic.com/en/articles/7996845-what-are-some-things-i-can-use-claude-for

Perplexity (Perplexity AI, Inc.)

What does it do with you data?

Perplexity terms itself as an ‘answers engine’ but is essentially a model aggregator — it offers it’s own Gen AI model (Sonar) plus ones from OpenAI and Anthropic. You can prevent Perplexity using your search data to improve it’s AI models in settings (turn off the “AI Data Retention” setting). It uses your approximate location (it doesn’t state how, but likely inferred from your IP address).

It does not pass your information to LLM third party providers. The Privacy Policy details more about third party providers usage data for the purposes of tailoring content or analytics. You can opt out of Google Analytics by using this browser add-on.

How long does it retain your data for?

No specific time is given — it’s Privacy policy states “as long as is reasonably necessary”. However, you can request to know which information Perplexity hold about yourself or for it to be deleted/corrected.

Uses: Search, chat, image generation, news

Cost: Quick search ($0/£0 p/m), Pro ($20/£15 p/m)

Try Perplexity here (no sign in needed):

https://www.perplexity.ai/

Read more:

• Pricing https://www.perplexity.ai/pro
• Models offered by Perplexity https://www.perplexity.ai/hub/technical-faq/what-advanced-ai-models-does-perplexity-pro-unlock
• Technical FAQs https://www.perplexity.ai/hub/technical-faq
• Data and third party providers https://www.perplexity.ai/hub/technical-faq/3rd-party-data
• Privacy Policy https://www.perplexity.ai/hub/legal/privacy-policy
• Getting Started https://www.perplexity.ai/hub/getting-started
• Help Centre https://www.perplexity.ai/hub/helpcenter
• Referrals Program https://www.perplexity.ai/help-center/en/articles/10354305-coupons-discounts

Gemini (Google)

What does it do with you data?

Gemini is angling to a become a digital personal assistant. It collects all your data from the conversation (files, images, recordings, your feedback, location info). Google are planning on expanding data collection to your Google ecosystem (from Gmail, Photos, Calendar, Search history and Youtube viewing) to help achieve their aim of a more personalised experience.

How long does it retain your data for?

All conversations are kept for 72 hours, as a minimum. You can customise your data retention settings in your Google Activity account.

Uses: Chat, images, video generation, coding, mobile assistant (Android phones), use the paid-for version with your Google Workspace account apps.

Cost: Free ($0/£0 p/m), Gemini Advanced ($/£19 p/m)

Try it here (no sign in needed):

‎Google Gemini

Read more:

• What can you do with your Gemini mobile app https://support.google.com/gemini/answer/14579631?hl=en&co=GENIE.Platform%3DAndroid
• Gemini Release Updates https://gemini.google.com/updates?hl=en
• Gemini Apps Help Centre https://support.google.com/gemini/?hl=en&sjid=8287867305078999873-EU#topic=15280100
• Gemini Apps Privacy Hub https://support.google.com/gemini/answer/13594961?hl=en
• Google Privacy Policy https://policies.google.com/privacy
• How Google Uses Location Information https://policies.google.com/technologies/location-data
• Google Activity https://myactivity.google.com/?continue=https://myactivity.google.com/product/gemini?utm_source%3Dhelp
• Google Privacy Check Up Tool https://myaccount.google.com/intro/privacycheckup?utm_source=pp&utm_medium=Promo-in-product&utm_campaign=pp_intro
• Gemini will soon tap into your Google Account data to become a more ‘personal’ AI https://www.androidpolice.com/gemini-personal-data/

Copilot (Microsoft)

What does it do with you data?

Microsoft say in their privacy policy they may use your data to develop and train AI models but not usually. Your data is kept confidential. Prompts are not available to other customers, are not used to train or improve any third-party products (OpenAI models).

Your data is also used for advertising purposes. You can switch off ad personalisation and download/delete your data in Copilot Account Privacy settings.

Microsoft uses both automated and manual (human) processing methods.

How long does it retain your data for?

It’s not entirely clear. At least one day, up to seven days for Enterprise accounts (I’ve contacted Microsoft to try track down this info for personal accounts — unfortunately their reply didn’t tell me what to expect for Individual accounts).

Uses: Chat, images, assistance with Microsoft 365 apps e.g. email prioritisation in Outlook and formula generation in Excel.

Cost: Copilot free (£0/$0 p/m) and Copilot Pro (£19/$ p/m)

Try it here (no sign in needed):

Microsoft Copilot: Your AI companion

Read more:

• Microsoft Privacy https://www.microsoft.com/en-gb/privacy
• Microsoft Privacy Statement https://www.microsoft.com/en-gb/privacy/privacystatement
• Microsoft Privacy FAQs https://www.microsoft.com/privacy/faq
• Copilot (feature overview and FAQ page) https://www.microsoft.com/en/microsoft-copilot/for-individuals/?ep=0&form=MA13LV&es=31#faq
• Ad Settings https://www.microsoft.com/en-gb/privacy/privacystatement#mainhowweusepersonaldatamodule
• Microsoft Privacy Dashboard https://account.microsoft.com/privacy
• Learn about Data Retention for Copilot and AI Apps https://learn.microsoft.com/en-us/purview/retention-policies-copilot

TLDR:

• Best free account for privacy Claude ‘Free’ plan
• Best value paid-for account Perplexity (access to Claude, ChatGPT and free $5 API credit p/m)
• Most potential for extensive data collection Gemini

Every talk I attended was engaging but I want to share 3 insights from 3 specific talks:

“Automation in the Age of Generative AI” — David Edmundson-Bird (Principal Lecturer, MMU)

1 Combine traditional automation with GenAI. Leverage the creativity and novel content generation of GenAI to compliment and fill the gaps of rigid rules-based automation, making a flexible tool that can tackle more than just process repetition.

2Map your workflow — identify and record the manual process, identify where no value is generated and determine where human judgement is absolutely needed, compared to where GenAI can contribute.

3 Consider security and data compliance. GenAI is leaky by nature so verify GDPR compliance and get to know how providers will handle your data e.g. OpenAI doesn’t reuse data from Teams or Enterprise plans.

“Preparing the Workforce for AI” — Laetitia Sterling (Senior Managing Consultant, IBM)

1 AI is about people, not just tech. Jobs are transforming, rather than disappearing. So continue upskilling employees, not just in AI tools, but in humans skills like communication, leadership and teamwork.

2 AI should augment human intelligence. Use AI to automate boring tasks while keeping interesting and fulfilling work human centred.

3 Build in trust and governance. Focus on shared values, not just rules. Embed ethical considerations throughout your organisation and implement AI with a bottom-up approach, not just top-down directives.

“Show Up, Wise up, Rise Up: Overcoming Self-Sabotage when Innovating” — Dani Wallace (Public Speaking Coach)

1 Develop your self-awareness about your work style (are you most Results, Details, Ideas or People-orientated?) & recognise your common stress responses (do you Fight, take Flight, Freeze or Fawn (people please?).

2 Craft a clean “vocal business card” using the format: “I am [name] and I [clear short version of job title], and I help [whom] so that [impact].

3 Define what success looks like and build support systems to get there — identify people who can mentor you, practice your new skills and consciously envision and embody who you want to become professionally.

I realised on my journey an emergent iterative process was occurring as I was progressing over time. If you want to teach yourself something new, here are 7 steps which might help you get started and create something new.

1. Set a goal (and know why it’s important to you)

My ‘why’ initially came from reading about Edward Snowden (the NSA whistleblower) in his autobiography. Concerned about the amount of data I was outputting and instantly losing control over, I decided I wanted to create my own tools.

So think about what you want to achieve and why. From this, I could develop some goals:

• Create my own server — this is a massive project, I have no knowledge of how. Let’s redefine it down to…
• Learn how to code — still a high goal, far too broad for now. Next!
• Could I build a personal portfolio website? — bit more specific, more manageable (if you consider a website can be one page of HTML).

So set a goal, break it down into smaller ones until you find something manageable that you can work towards, ideally something that you can do by learning one skill. If you can find some way to creating something of your own, it’s a great feeling and will motivate you to create (and therefore learn) more.

2. Choosing your tools

There’s no “perfect” learning resource. The only way to find out which you like best is to try them all out and notice what sticks. Consider trying:

• YouTube tutorials — especially for more practical skills!
• Apps — quality really varies with these on the Google Play Store.
• Online courses — different teachers have different styles so try a few out.
• Use AI — as a teacher, study planner, road map, dictionary and mentor.
• Books — good for getting to know something in-depth.
• Academic journals —there’s Open Access resources e.g. ArXiv

AI prompts to try:

• “Play the role of an experienced [software dev, financial adviser, insert your desired expert role here]. I’m your mentee and I want to learn to [code, teach, any skill you want]. Advise me on where I can begin to learn this skillset.”
• “You’re a Physicist. Explain quantum theory like I’m five.”
• “I am a UK poet; I’ve started to draft a book of poetry. Suggest events/conference in 2025 I could visit in person near Birmingham to enable me to network with publishers.”

3. Learn the theory and apply the learning.

Start learning and keep progressing through the resource, even if you’re confused. Then try apply it.

You don’t truly understand something until you use it. I usually completed a course then I would code something afterwards to cement my new knowledge.

Create your own project or Google for ideas of ones you can copy. The result being finished or perfect or original doesn’t really matter, what’s important is giving yourself an arena to apply yourself.

Ultimately get a feel for your new skills and knowledge on an intuitive level as well as prove to yourself you can do it. If you fail, it’s not a big deal. Be aware of where your skill gaps are and make a note to work on it in the future, if you want.

4. Growth Mindset (or learning to fail)

This is the most difficult as an adult, we all get good at our area of expertise so going back to basics in a new area is hard. Accepting when you take on something brand new, you will suck until you don’t suck anymore.

Some of the goals you set initially and had to dump, you can now pick up again, which is the exciting feeling of progress. If you find yourself struggling to finish (because of time not boredom), set yourself a deadline and remember your ‘Why?’ from step 1.

Study tips:

• Try focus on just one skill or resource at once — don’t spread progress too thinly.
• Consistency over time is more important than each individual effort
• Find a course or resource to help you learn just one skill at once.
• Give yourself permission to drop a course if you hate it.
• Extend the deadline if you need to but be aware if you feel guilty as it’s probably a sign you do want it, you just need to reprioritise your time.
• Give yourself small rewards for hitting milestones.

5. Find a community (or be good at problem solving)

A community can be in real life, online or make a network of contacts yourself. A community can be as small as a few online people to chat about problems and ideas with, as well as ask for help. It could be as big as a global Discord special interest group. You don’t have to do this step, but many hands do make light work.

What could take you days to resolve, can be as simple as asking one person for help for 10 minutes. Even knowing people with similar knowledge can be advantageous e.g. coding concepts and skills can transfer between languages. I’ve solved R problems without ever learning that coding language because it’s so like Python.

Don’t forget about AI. If you learn to prompt efficiently, it can act as a soundboard and ideas generator and even a coder, first drafter, project planner, food shopping list organiser…

6. Take notes and repeat steps to 1–6

‘Practice makes perfect’ as my nana used to say. It’s hard to build a learning habit, some people find 20 minutes a day is enough, others want to get stuck in for a 3 or 4 hour deep focus session.

I used a two-pronged approach — Linkedin Learning courses (I aimed for an hour a day which is easy when you’re having fun) and an app I used on my daily commute for 20 minutes (called Mimo) to review and reinforce knowledge I’d learnt on my Linkedin Learning courses.

While doing this, also take notes. Note down ideas to create, ideas you might create in the future, what you’re learning, what you’re stuck on. Make lists, draw diagrams. If you’re not sure what to do next, look on YouTube for ‘roadmaps’ and plans from people who have done it before or think about your ‘Why’ from step 1.

7. Pause. Take time to appreciate your achievements

Continuously reflect for few minutes each week. Appreciate achieving your mini-goals. If you ‘fail’, take the time to consider what you’ve learnt while striving for your goals. If you achieve your overall aim and enjoy the process, extend your plans.

The beauty of self-directly learning is that YOU control your own path.

Once I create a website, I had learnt enough to know my primary interest didn’t lie in web design right now. I ultimately took to a programming language called ‘Python’ which is about more than just application development. I ended up exceeding my target and completing 14 or 15 courses in 2023— seemingly crazy now! In 2024, I did return to web dev.

About Sheldon

I’m an Library and Discovery Adviser at Manchester Metropolitan University. Currently, I’m working towards creating a Virtual Bookshelf app.

Connect with me on LinkedIn @ https://www.linkedin.com/in/sheldon-korpet/

Actionable insights and tips:

• Write a ‘brag doc’ to make advocating for yourself easier — at the end of each week, calendar 10 minutes to bulletpoint your achievements for the week — what you improved, the impact you had, time saved, revenue boosted, as well as praise (screenshot compliments, thank you emails and testimonials)
• Present to your leaders, not just your line manager - Drop them an email, share insights in meetings.
• Seek feedback — not just from your manager but project leaders, subject experts and peers.
• Seek mentors — external mentors could too, so look in communities of practice, networks, meet ups. Ask for help (“I never found anybody that didn’t want to help me…you gotta act and be willing to fail” — Steve Jobs)
• Think from other perspectives — consider others roles and their priorities, adapt to their working style and be proactive in suggestions to meet them where they are.
• Create psychological safety for others — to get people to open up, suggest ideas and give feedback, it might help to get to know others on a deeper level. Make an effort to talk to people daily if you work in the same location and create/attend team socials.
• Learn AI prompting and experiment with tools — different LLM’s have different strengths so be mindful of which tool you need to use in which context

When you doubt yourself, remember:

• For imposter syndrome — Ask yourself “where am I adding value? Am I visible?” Go looking for your own motivation, inspiration, direction and recognition.
• For perfectionism — you don’t need to be an expert, you just need to know enough to do what you need to do. Learn one small thing every working day and in a year you’ll know over 200 new things.
• Starting a new team/project — new teams do not have established ways of working, so be transparent and reflective. Try to evaluate what is working and be reflective about what isn’t.

Useful tech tools to try:

No/low code:

• Google AI Studio (free Gemini API key)
• LLMs — Claude, Deepseek, ChatGPT, perplexity.ai
• AI Agents — Bolt, Mindstudio
• Accessibility tools — Google Lighthouse, Microsoft Accessibility Tools
• More AI tools here: https://github.com/jamesmurdza/awesome-ai-devtools

Technical/coding tools:

• AI code editors — Cursor, Trae, Intellij IDEA, Windsurf
• AI coding assistants — Tabnine, Devoxx Genie (https://github.com/devoxx/DevoxxGenieIDEAPlugin), Devin AI, Github Co-pilot
• Replit (Open source version, Sandbox: https://sandbox.gitwit.dev/)
• AI CLI assistants — Ada (https://github.com/iamlucasvieira/ada-cli), Aider (https://github.com/Aider-AI/aider)
• Debugging tools — Chrome DevTools, VisBug
• VS Code extensions — Gitlens, Indent-rainbow, Prettier

Photos by Wilhelm Gunkel & Recha Oktaviani on Unsplash

What have I learnt since April 2023? What have I built?

In April and May I played with number generators, building in Python a number guessing game, a lotto number generator (with a tkinter GUI) and a compound interest calculator (a mess, not the math I borrowed, just my coding). Building useful tools was good fun and playing with GUIs stood me in good stead later to create my first original program later and the basic stats became a good introductory confidence building exercise for the Machine Learning course.

May 2023 — Software Development Life Cycle (Linkedin Learning)

I realised by building projects I had no idea what I was doing when organising myself and creating programmatic logic and breaking features down to build up a complete tool. This course exposed me to:

• MoSCoW prioritisation for features (Must have, should have, could have, will not have)
• Scrum principles (product backlog, sprint backlog, sprints, product increment with roles — product owner responsible for choosing from product log the work to complete, scrum master coaching and motivating the team and team members developing, testing and documenting)
• Kanban boards — for organising and tracking tasks.
• Pair programming method — one person coding, one helping. Second person views and asks questions/makes suggestions before switching.

Jul 2023 — Django Essential Training (LinkedIn Learning)

Django was perhaps the most confusing thing I’d encountered since my intro to algorithms and navigating the file directory. I build a website which I could view locally via a port on my computer. I developed an appreciation of why Django had been able to scale for Instagram, Dropbox and Pinterest but ultimately decided it might not be my tool of choice for a personal portfolio.

Aug 2023 — Python course (Mimo)

Throughout 2023 I utilised the Mimo app to review Python knowledge I’d gained through earlier courses on my daily commute. I felt this was a good move in retrospect. It solidified (and reminded me of) concepts as well as providing an alternative explanation for the same concepts which helped.

Sept/Oct 23 — Making networks and teaching

I hadn’t been completely alone in my journey this whole time. My colleague had also picked up a coding language around them same time as I had. Where I had taken a knowledge-based approach to learning and applying concepts, she had taking a project-based approach, building something she was interested in and learning as she went. It was great to discussed our struggles and through our experience swapping (and our fascination/fear of AI) she was inspired to found a Community of Practice at work.

I later went on to teach 10 of my colleagues the very basic principles of coding via Python. It was my first proper experience of teaching and it challenged me to reflect back to the very beginning and amazed me how much I’d managed to absorb by this point. It’s proven be a very small catalyst and an excellent knowledge sharing (and time saving) opportunity.

Oct 2023 — SQL Essential Training (LinkedIn Learning) & SQL course (Mimo)

Over the summer I met a wild duck with a cataract. Over time with frequent visits we built what I saw as an interspecies friendship based on trade — sunflower seeds for chest strokes. However, I’m sure my feathered-friend thought of me as the world’s first Duckaroo deliverer, as he would later learn to jump out of the water to collect his food.

I decided to build a local Python program with a GUI which have a database on the backend to capture and store time, date and location data . After a couple of SQL courses, Donald’s Diary was born.

The UK has a legal duck hunting season where some of the most beautiful ducks can be culled including the Tufted Duck, Pintail and Widgeon (see ‘Quarry species’). It’s arguably a necessary process in Mallards as overfeeding leads to excess males which puts stress on females come breeding season but that also squarely put Don in the firing line. I wanted to build something local on my machine in the spirit of taking back control of personal or sensitive data.

This was the first time I used a conda environment to control the environment for the PySimpleGUI and SQLite3 libraries and create a program which captured the data I input and allowed me to backup data via a csv export:

Goal complete

By the end of October I’d achieved the goal I’d set out to complete — study 12 courses within the 12 months of 2023. Instead of feeling elated, I felt deflated. What do I do now? I felt I had little direction anymore.

The importance of a network shone through here and gave me perspective. You’ve done well — why stop now? There’s always a next step and the great thing about learning and creating is that you can decide the direction.

What are you still curious about? What can you create now?

Nov 2023 —Machine Learning (Code First Girls)

GPT’s had played a bit part in my coding journey to this point. I’d even hosted one locally thanks to H2OGPT and open source LLM models via Hugging face (this project was scuppered by a lack of VRAM).

I was curious — what made this technology work? I decided to take an introduction to ML, learning about supervised and unsupervised learning, linear regression, k-nearest neighbours with scikit-learn, understanding bias and algorithm evaluation. It was basic but insightful.

I also learnt to launch Jupyer locally and implemented a few basic algorithms. It’s a specialism in itself and something I’d like to continue to explore in more depth in the future.

Dec 2023 — Collaborative Coding with Git (FutureLearn)

This whole year I’d been using Git to push to remote GitHub repos. However, when myself and a few colleagues got together to use it to write and share code, we were instantly confused.

This course made me realise how powerful Git is not only for developing code but collaborating, staying organised and getting an overview of how repos change (including your own historic code). I made use of GitKraken and viewed some of my old code to see where I could improve my use of versioning commit messages and (lack of) branching for feature development.

Jan 2024 — Learning Docker (LinkedIn Learning)

I had previous experience of using env’s with conda for Donald’s Diary. This is a great tool for managing package versions. If you need a little more, you can create an entire OS and its dependencies, providing a consistent environment across different systems.

Through this course, I have acquired an understanding of essential Docker concepts and their associated commands. From setting up Docker on a Windows machine to creating images and managing containers, running applications, and troubleshooting.

I’ve also found out when to use Docker Compose and when to consider Kubernetes, and how to choose the best images and manage versions on Docker Hub.

“Are we there yet now?”

I set the 12 courses in 12 months goal because I wanted to create something. I set and exceed a goal (15/12 courses completed!) and learnt more than I imagined. I’ve built websites, created programs and called from APIs. As well as got my head around a lot of new tools and new ways of thinking about, and conceptualising, problems.

Despite having few expectations when I started coding, I made new connections and good friends who supported me along the way. In return, I have tried to share my knowledge to help others begin their coding journey. Overall, 2023 was a significant year, marked by the achievement of my learning goals, creating my own programs, and the valuable relationships I formed along the way.

Through my coding journey I’ve tried two bootcamps and a Python course IRL. However, despite the opportunities given to me, coding never ‘stuck’. To be frank, every time someone opened their mouth to simplify something, I’d become more confused.

This story is about success as a result of repeated failure. A sort of cyclical, exponentially growing, purposeful failure. A failure which cycles between success and failure at such a rapid fire fast pace, the force of the falls becoming so strong it bounces you back up to standing and you’re simply ready to learn, fail and fall again.

Despite bouncing along like a sprite in a game glitch, I’ve learnt a lot through the process of learning to code independently. Here’s my road map, the lessons I’ve learn over the last 6 months and what I’d do differently if I could start all over again.

Chapter 0: Foundations of User Experience Design (June 2022)

Before I started coding, I considered becoming a UX designer. I love tech which is designed well and just works. After this course I realised it didn’t appeal to me. However, it gave me an insight in to good design for users and made me realise I had to desire to create something tangible…

Chapter 1: Making Sense of the CSS Box Model (November 2022)

I took this course purely because I had an information need. I started to build a website but had no idea how to use CSS to alter the HTML elements vertically and horizontally. It was after the frustration of combing this course for the right command to save me that I realised:

It’s easier to learn and understand theory/functions before trying to apply them.

Chapter 2: Programming Foundations (December 2022)

I had committed (again) to the idea of learning to program by this point and I took a step back to the basics. My aim was to get a broad understanding of some principles. As a result of Annyce’s enthusiastic classes about variables, data types, and if statements, I wrote my first original Python script, an interactive story. I realised then:

Coding is limited only by my imagination (and lack of coding knowledge).

Sheldon Korpet BA (Hons), MSc on LinkedIn: Certificate of Completion

Chapter 3: Learning Python (January 2023)

The foundations course had shown me something I’d never felt before — Python was almost limitless, if only you knew what to type and could import the right libraries. I started a reference document for loop and function examples, listing the technical terms and definitions I was learning. I’ve carried on adding to this document over all the courses:

Now armed with the Prettier extension on VS Code, I felt like I was painting in words. I loved writing more code, seeing the different syntax elements cover the screen with strokes of bold and bright colours:

I started to create a cataloging tool that suggested FAST Subject Headings from an API. I still had some functions to add but I was excited. Having no idea about packaging programs, I improvised using a library called Pyfiglet to create some hideous ASCII terminal text and a file shortcut with a changed icon to create my first “program”, FAST-inSH:

I asked the API owner about the response from the server and in passing the supplier kindly pointed out if I was to achieve my end goal of pulling both the subject heading AND unique ID, I’d need to use a different API. I’d used the wrong API! My next useful lesson learnt:

Everything has docs. Always read the docs, then read them again.

Chapter 4: Python: Working with Files (February 2023)

This was where the Dunning-Kruger effect became very real. Learning to navigate in the terminal felt like an insurmountable challenge to me. After failing to read the API docs I’d started to slide off Mount Stupid and this course plummeted me to the depths of Despair Valley.

I didn’t just realised the mountain before me was huge, I knew it intimately. I hated every second, feeling lost and stupid. The only minuscule achievement I felt during this three week course was calling a JSON input from an API and output it as a CSV file. I seriously considered quitting. Afterwards, as I picked my self off the floor for the nth time, a new lesson:

One day, this will be easy. Just not today.

Chapter 5: Python Object-Oriented Programming (Feb 2023)

Throughout the journey to date I found it extremely hard to think like a programmer. My logic was dire and my code messy. I had hoped organising files would have worked to my strengths but I couldn’t see a visual representation of the file structure which left me feeling disorientated.

I needed to work on structuring something which was in front of my eyes, all in one place, in the hopes it could boost my confidence. Learning about classes really was a huge turning point for me. Organising code felt like it came naturally but it simply build on the principles and foundations I had used so often as a librarian — everything has a category and that categories has items.

I took my original text adventure script I wrote after Course 2: Foundations and re-wrote it using OOP. The result was so satisfying and I utilised my new knowledge of f strings to make the story come together at the end of user input. I send it to a friend to test and he instantly broke it. I was simultaneously disappointed and elated because it was a lesson learnt:

You can’t predict or fix all the bugs in your program first time.

Chapter 6: Programming Foundations: Beyond the Fundamentals (March 2023)

This course was a great recap. To my absolute joy, I realised I’d accidentally taught myself a lot of the content of this course. It served to boost my confidence that I was on the right track and understanding the theory.

I thought I was on the Way to Enlightenment (see Dunning Kroger again). I hadn’t realised it yet, but I’d left the Valley of Despair and climbed right back on to Idiot Mountain. The latest lesson occurs:

Be proud of your achievements, not overly confident of your abilities.

Chapter 7: Python Data Structures and Algorithms (March 2023)

As fast as my confidence had grown, the ground rose up to slap me in the face again. Or should I say, I’d tripped and landed face first. I felt disorientated and confused because data structures are disorientating and confusing; I was stuck in file directory hell all over again, back in coding despair and frustration.

But I’d been here before. While navigating file structures had been a struggling previously, I’d since grown in confidence with the command terminal. Instead of wasting time focusing on my inevitable ignorance of the new topic, I powered through the course, headbutting the floor on repeat to stand up and take on the next bit. Another lesson learnt:

Being a beginner is always hard but not insurmountable.

Chapter 8: Introduction to Career Skills in Software Development (March 2023)

After writing a comprehensive list of what topic I could study next I promptly ignored said list and began to wondering, what would it be like to do this as a job?

I already have a lot of transferable skills (problem solving, team working, research, attention to detail, collaboration and communication). Tech offers a balanced mix of team and independent working. The software dev cycle seems like it could give lots of opportunities for finishing projects and launching products, which is not a completionist satisfaction I get to experience much in my current job.

“Are we there yet?”

The rate of failing is now sometimes lower when I code. As I begin to incorporate the coding concepts I’ve learn in to new things, I find I have more first time successes and debugging leads to faster fixes.

My aim is to keep learning though. The failure and frustration are a symbol of the journey to me now. Maybe I’m on my way along the ‘Path of Enlightenment’ after all…

The last 6 months — what would I have done differently?

• Started using Git bash/Github sooner — be able to revert back to previous versions of code.
• Codewars — Self-testing the ability to apply knowledge to small challenges and so insightful to help spot weak areas.
• TLDR — it’s interesting but reading it daily takes much time. Maybe a weekly tech news podcast would have been better.
• Mimo — It’s great app for re-capping knowledge on the commute but I wouldn’t use it as the primary method of learning again.
• Data structures — I’d have benefited from looking at a YouTube road map sooner to get an idea of which topics were advanced/to be avoided.
• Boot camps — all the rage in tech and they’ve supported a lot of people but for other they aren’t the best way to succeed. I wish I hadn’t wasted time feeling like boot camps were the only “legit” way to learn to code.

Part 2 here — post-ChatGPT launch: https://medium.com/@sheldonkorpet/my-programming-journey-pt-2-beyond-beginner-99905bd0a5d6

Your local University Library’s will likely let you register as an alumni or walk-in member of the public. However, you’re allowed to access a very few databases and only loan a token amount of books — which is great if you live locally. However, many people don’t or just want to save time and effort. So, how do you find open information online? Try Open Access information!

‘Open Access’ was created for the public — it’s a way of licensing academic information (like journal articles and some textbooks) to make it freely available, for everyone.

As an Information Professional, here are my top five resources I recommend you should try out to help you access digital resources by topic.

This is a search engine that makes academic information both discoverable and easier to digest. Use it at the start of a search, instead of Google.

Advantages:

+ user friendly interface

+ evidence based quick overviews

Disadvantages:

— new so there’s likely some bugs to iron out

— not clear what information from external sources is updated automatically

A visualisation tool for literature, demonstrating topics and the relationships between them. Use it to get an overview of the most ‘relevant’ ares of a topic and papers related to those concepts.

Advantages:

+ generates visualisations for your search terms

+ it has an option to visualise results of searches just from PubMed

Disadvantages:

— still in development

— it only analyses the first 100 papers based on relevance ranking

A website which allows you to search for an Open Access version of a paper using it’s URL (web address), DOI (permanent Digital Object Identifier) or title. You can use it when you’ve found a journal article you want to read, but the publisher tries to charge you to access it.

Advantages:

+ easy to use

+ no installing or configuring, unlike Unpaywall

Disadvantages:

— success isn’t guarenteed (the tool relies on academics submitting a copy in to a repository, like White Rose Research Online)

A vast collection of repositories. Repositories are places where institutions store publications created by their academics. CORE allows these to be simultaneously searchable through a single interface. It can be used when you have a keyword search and want a more in-depth, systematic overview of a topic or problem.

Advantages:

+ search a lot of credible information, fast

+ there’s an API for text mining

Disadvantages:

— you aren’t searching ALL the repositories that exist in the world. It’s possible you will miss a source

A collection of open access journal titles, searchable by title or article. Could be use it to follow a specific journal that consistantly produces articles about a topic you’re interested in staying up-to-date with.

Advantages:

+ all of the journals included are Open Access — no paywalls

Disadvantages:

— there is a small chance of encountering a predatory (scam) journal - however, each journal does undergo over 40 checks before it is listed

And finally, here is a bonus resource to watch out for in the future…

There’s a growing trend towards publishers releasing textbooks, free for anyone to access online. This is one way to search for them. Use this tool when you want to find a comprehensive introduction/overview of a topic or subject.

Advantages:

+ avoid annoying previews, this is the whole textbook, for free!

Disadvantages:

— the idea of Open Access textbooks is still a fairly new movement, so there’s a limited selection.

— you will have to buy the book if you want to own a complete physical copy of the material. Printing could infringe UK copyright law.

No one is perfect — if I’ve missed a good tool, you can share your knowledge with me by tweeting at SheldonKorpet.

About Sheldon Korpet

I’m an Information Officer at the University of Sheffield. Currently, I’m working towards becoming a Chartered member of CILIP and I also volunteer with Code Club.

Prior to this, I received an MSc in Digital Library Management in 2017 after undertaking a Graduate Traineeship at Manchester Metropolitan University.

I’m happy for you to share my work under CC BY-NC 3.0 — learn how to attribute someone using this useful guide from the Creative Commons wiki.

What is Text Data Mining (TDM)?

TDM is a way of analysing data computationally. It can be used to look for themes and sentiment within documents or to compare documents’ word usage or sentence structure to determine similarity.

Why is TDM Important?

Scholarly publications are increasing at an overwhelming rate. TDM has helped the researchers we have interviewed deal with increasingly large amounts of information by examining it in new ways and deal with information overload. The ability to examine huge data sets has also enabled the study of social media data which would have been vastly time-consuming or simply impossible to analyse.

Who Uses TDM?

On undertaking our interviews we were able to find researchers from all five of the University of Sheffield’s subject faculties. These methods are being used widely, beyond computer science. However those researchers interviewed often spoke of a need for programming or statistical knowledge to be able to exploit the technology to its fullest extent.

How Could an Academic Library Support TDM?

Academic libraries already host information and digital literacy skills programs, maintain publisher connections and content collections. In addition they have copyright specialists and have subject-neutral spaces. These key assets could help researchers access the information they need and counter the legal challenges of TDM to support its growth.

Read the report to learn what we recommended the University of Sheffield Library could do to support TDM in its institution.

A Practical Class Project

We decided to release this report in to the wild thanks to the recommendation of our supervisor, Dr Andrew Cox, and our interview participants — many of whom found the end result of interest.

Dr Erica Brown is working in Scholarly Communications at the University of Manchester.

Sheldon Korpet is an Information Officer in the School of Health and Related Research, University of Sheffield.

Bálint Csöllei is a Freelance Information Professional.

Useful resources

Fact Sheet: Text Mining — NLPN
Text & Data Mining — University of Cambridge Library LibGuide