In Mac or Linux, open the terminal using the following command to check what jobs are running in the background

crontab -l

Now, to edit the list type the following command

crontab -e

You can then give the path of your python script file to run at a specific time by giving the time particulars. Now enter the following and press CTRL+O and CTRL+X to save and exit.

Presently I have two jobs scheduled to run in the background as can be seen from the below screenshot

The first job will run at the 15th minute of 11th Hour on the 27th of the month. This job periodically scans the contact details of field offices and store them in a csv file.

The second job runs at the 15th minute of every hour to check the weather details of various places.

Structure of cronjob:

For Windows, search for Task Scheduler.

1. Open the program.

2. Now, on the right panel click on “Create Basic Task”

3. Next select how often do you need to run the script.

4. On the next screen, select date and time on which you want to run the script

5. Next, select “Start a Program”

6. Browse for the script to be executed

7. Select the script and click on ‘Finish’ to schedule the task

Happy Coding!

Easiest way to Run a Python Script in the Background was originally published in Analytics Vidhya on Medium, where people are continuing the conversation by highlighting and responding to this story.

The simple and straight-forward answer is “No”.

Some of the most notable features that may be considered as breaking backward compatibility are:

• print is now a function, not a statement, and using it as statement will result in an error,
• various functions & methods now return an iterator or view instead of list, which makes iterating through their results more memory-efficient (you do not need to store the whole list of results in the memory),
• cmp argument for sorting functions like sorted() and list.sort() is no longer supported, and should be replaced by key argument,
• int is now the same as Python 2.x’s long, which makes number processing less complex,
• / operator is now an operator for true division by default (you can still use // for floor division),
• text in Python 3.x is now Unicode by default,
• True, False and None are now reserved words (so you are not able to do True, False = False, True

More details can be found on the following link:

https://docs.python.org/3/whatsnew/3.0.html

As per Wikipedia, Data science is a multi-disciplinary field that uses scientific methods, processes, algorithms and systems to extract knowledge and insights from structured and unstructured data.

Literally speaking, we all are data-scientists. We read about lots of data be it in the news, or our office works, colleges, schools and come to some sort of conclusion based on it. However, when we are unable to conceive and process large amounts of data with our bare eyes , we have to take help of machines and this process is known as “Data Science”. Data science is the precursor to Machine Learning and Artificial Intelligence (A.I.)

For learning data science with reference to machine learning and AI, you need to have a good knowledge on Python Language. Python is the most effective and efficient language to perform large-scale scientific operations on data. It has lots of libraries specially for data-science.

You can learn Python in 1(One) month from Udacity for free. They also have paid courses as well but those aren’t very different from the free ones. The only difference is that you have to submit a project at the end of the course for getting certification.

Once you are comfortable in python, you can start learning Data Science right away. I had learned from Udacity. Their course is nicely structured and the instructors are well versed with the subject. There are many real world problems with data-sets to apply your data science models.

You also need to learn Algebra in order to learn data-science. Various statistical methods such as linear regression, correlation, bayes method etc. are useful for performing data-science operations.

But whether you learn in 6 months or 1 month, regular practice is the key for retaining the learning. Keep exploring new data-sets with the data science models regularly. This will help you to grasp the subject in a more efficiently.

Good luck.

Background

Load forecasts are extremely important for energy suppliers and other participants in electric energy generation, transmission, distribution and markets. Accurate models for electric power load forecasting are essential to the operation and planning of a utility company.

A conventional electricity demand forecasting requires analysing the following types of variables:

• Calendar data: Season, hour, holidays, etc.

• Weather data: Temperature, humidity, rainfall, etc.

• Company data: Price of electricity

• Demand data: Historical consumption of electricity.

But, due to growing use of distributed energy resources (DER) such as roof-top solar, wind, biomass etc., the power demand scenario for the distribution utilities are changing. With more and more number of consumers opting for such own generation options, the utilities would need a smart demand forecasting and allocation technique using the latest tools available to optimise cost and provide reliable services.

Way Ahead

To overcome such difficulties, new techniques for demand forecasting and load allotment could be devised using IT tools such as Big Data and Machine Learning (Artificial Intelligence).

Big Data — For information Gathering and Assessment

Big data is a term used to refer to data sets that are too large or complex for traditional data-processing application software to adequately deal with.

Big Data will be able to capture, process and analyse the following information:

1) Historical power demand information for all consumers

2) Distributed generation information for all consumers

3) Weather forecasting information (provided by the agencies)

4) Load Growth

5) Consumer Mix

6) Special Factors such as Festivals etc.

7) Power prices at the IEX (Indian Energy Exchange) in real time

Machine Learning (ML) — For Analysis and prediction

The data gathered through Big Data System could be brought together in a single predictive ML model, so as to discover associations between all the above variables. This will give us deeper understanding on the causes of demand and will allow better decision making. This involves the accurate prediction of both the magnitudes and geographical locations of electric load over the different periods of the planning horizon.

On the basis of information provided by the ML model, adequate planning could be done by the utilities to purchase power accordingly. This will also help in better load allotment to various locations on the basis of their predicted demand considering the above factors.

About Machine Learning

Machine learning (ML) is a field of AI (artificial intelligence) that uses statistical techniques to give computer systems the ability to “learn” (e.g., progressively improve performance on a specific task) from data, without being explicitly programmed.

Broadly, Machine Learning falls into two major categories

- Supervised Machine Learning

- Unsupervised Machine Learning

Supervised Machine Learning

- Supervised learning is the machine learning task of learning a function that maps an input to an output based on example input-output pairs.

- A supervised learning algorithm analyses the training data and produces an inferred function, which can be used for mapping new examples.

- Example : Association between weather information (such as temperature) and power demand. By varying the temperature variable, we can assume that the power demand would vary.

Unsupervised Machine Learning

- Unsupervised learning is a branch of machine learning that learns from test data that has not been labelled, classified or categorised.

- Instead of responding to feedback, unsupervised learning identifies commonalities in the data and reacts based on the presence or absence of such commonalities in each new piece of data.

- Example: Sometimes, the power demand drastically falls or increases without any significant variance in the associated factors. Unsupervised ML can learn from such scenarios so that such scenarios could be tackled efficiently in future.

Advantages:

- Machine Learning Models could anticipate customer response and changing attitudes about energy consumption by looking at all types of factors.

- The results are improved forecast accuracy, which means better information to decide what the best course of action is.

- With such ML based models put in place, this would ensure smart decision making in power purchase which would save in cost incurred.

- Efficient load allocation would result in better customer satisfaction and improving system reliability

- ML model could automatically correct the forecasting techniques to keep track of changing conditions

- Over time, such models could automatically determine the various load growth centres of the utility and can help in planning for infrastructure upgradation / addition.

Challenges:

- Deploying Big Data systems is capital intensive as they require high capacity servers

- Historical information on load growth pattern in different parts of the state

• Accuracy of Historical information on power availability

Reference:

Arunesh Kumar Singh, Ibraheem, S. Khatoon and Md. Muazzam, “An overview of Electricity Demand Forecasting Techniques” Vol.3, No. 3, 2013- National conference on Emerging Trends in Electrical, Instrumentation & Communication Engineering