Here is an intermediate Python project that combines web scraping and data analysis: A Web Scraping and Analysis Tool for Cryptocurrency Prices.

Created by pawan natekar

Project: Cryptocurrency Price Scraper and Analyzer

Objective:
To create a Python script that scrapes cryptocurrency prices from a website and performs basic data analysis and visualization.

Requirements:
- `requests`
- `BeautifulSoup`
- `pandas`
- `matplotlib`

Steps:

1. **Setup and Installation:**
   Make sure to install the required libraries using pip:
   ```sh
   pip install requests beautifulsoup4 pandas matplotlib
   ```

2. Web Scraping:
We’ll scrape cryptocurrency data from a website like CoinMarketCap or CoinGecko.

3. Data Storage:
Store the scraped data in a Pandas DataFrame.

4. Data Analysis:
Perform basic data analysis like finding the top 5 cryptocurrencies by market cap.

5. Data Visualization:
Create visualizations to represent the data.

Code:

```python
import requests
from bs4 import BeautifulSoup
import pandas as pd
import matplotlib.pyplot as plt

# Step 1: Web Scraping
def scrape_crypto_prices(url):
    response = requests.get(url)
    soup = BeautifulSoup(response.text, 'html.parser')
    
    # Assuming we're scraping from CoinMarketCap
    table = soup.find('table', {'class': 'cmc-table'})
    rows = table.find_all('tr')
    
    data = []
    for row in rows[1:]:
        cols = row.find_all('td')
        rank = cols[0].text.strip()
        name = cols[1].find('p', {'class': 'coin-item-symbol'}).text.strip()
        price = cols[2].find('a').text.strip()
        market_cap = cols[6].text.strip()
        data.append([rank, name, price, market_cap])
    
    return data

url = 'https://coinmarketcap.com/'
crypto_data = scrape_crypto_prices(url)

# Step 2: Data Storage
columns = [’Rank’, 'Name’, 'Price’, 'Market Cap’]
df = pd.DataFrame(crypto_data, columns=columns)

# Convert Price and Market Cap to numeric values
df[’Price’] = df[’Price’].replace(’[\$,]’, '’, regex=True).astype(float)
df[’Market Cap’] = df[’Market Cap’].replace(’[\$,]’, '’, regex=True).astype(float)

# Step 3: Data Analysis
top_5_cryptos = df.sort_values(by=’Market Cap’, ascending=False).head(5)
print("Top 5 Cryptocurrencies by Market Cap:")
print(top_5_cryptos)

# Step 4: Data Visualization
plt.figure(figsize=(10, 6))
plt.bar(top_5_cryptos[’Name’], top_5_cryptos[’Market Cap’])
plt.xlabel(’Cryptocurrency’)
plt.ylabel(’Market Cap in USD’)
plt.title(’Top 5 Cryptocurrencies by Market Cap’)
plt.show()
```

Explanation:

1. Web Scraping:
- We use `requests` to fetch the webpage content.
- `BeautifulSoup` helps in parsing the HTML and extracting relevant data.
- The script extracts rank, name, price, and market cap of cryptocurrencies from the table.

2. Data Storage:
- The scraped data is stored in a Pandas DataFrame for easy manipulation.
- We convert the price and market cap columns to numeric values for analysis.

3. Data Analysis:
- We sort the cryptocurrencies by market cap and select the top 5.

4. Data Visualization:
- We use Matplotlib to create a bar chart showing the market cap of the top 5 cryptocurrencies.

This project helps you learn web scraping, data manipulation, and visualization in Python. You can extend this project by adding more features like scraping historical data, performing more advanced analysis, or integrating with a database.