Stories by Deepanshu on Medium

Google Summer of code at Debian Final Report

Deepanshu — Thu, 11 Oct 2018 16:26:26 GMT

Google Summer of code 2018

Organization — Debian
Mentors — Dashamir Hoxha , Akash Shende
Project Abstract — Link
Github repository — Virtual LTSP Server

Introduction
Final Summary
Deliverable
Weekly reports & Blog posts
Other contributions
Thank you

Introduction

Virtual LTSP server project automates installation and configuration of LTSP server with vagrant. It is the easiest way to create LTSP setup. We have developed the project to do the same for Linux mint 19 and Debian 9. We also created several scripts for testing, create ltsp client, manage accounts, etc. Also created packer scripts to create vagrant boxes that we will use in the project.

Final Summary

Google Summer of Code was a great opportunity to work with really smart and amazing people. I learned a lot in the process. It took my understanding of Vagrant, bash-scripting, packer, ltsp, Debian packaging to a whole another level. I started with a basic provisioner script to install ltsp to vagrant box. Then we make several improvements in it.

Later made several features and improvements in it. After that a major issue was the client was unable to boot. In order to solve this issue I searched through all the content on the internet about ltsp. Even asked the active developers of ltsp on how to fix this issue. They have been working on ltsp since 2006 but they haven’t encountered this problem yet. After struggling for lots of I solved it! Looking at the complexity and the time it took mentors told me to write a separate blog post about it. We have also created Virtual ltsp server for Debian 9. Also one for linux mint 19.

I had to create a new linux mint vagrant box with xfce. Which was really fun. I also automated its creation with packer scripts. We also did port from Edubuntu packages to Debian. It is locally built and installed via a small script. In the end, we added features like automatic login, guest login, and several scripts to optimize the workflow for the user. This was a really short summary of the work done. More details can be found on the weekly reports.

Deliverables

Virtual LTSP Server

docker-scripts/dev--LTSP

Pull request made

docker-scripts/dev--LTSP

Commits made

bionic branch —

buster branch —

Ported edubuntu packages from ubuntu to debian

Documentation

docker-scripts/dev--LTSP

Weekly reports & Blog posts

Week1:

GSoC weekly report of Deepanshu Gajbhiye week 1

Week2:

Week3:

Week4:

Week5:

Week6:

Week7:

Week8:

Week9:

Week10:

Week11:

Week12:

Other contributions

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=905006

Thank you

I am very thankful to Google and Debian for accepting me to Google Summer of Code. Working for GSoC was an amazing experience. I will definately participate next year as well.

Thank to my mentors Dashamir Hoxha and Akash Shende for their solid support, quick response, patience and trust on me.

Thank you Daniel Pocock for encouragement and Big thanks to Debian, ltsp, Vagrant community for being very helpful.

Week 12: Google Summer of code with Debian

Deepanshu — Sun, 05 Aug 2018 18:02:50 GMT

Here is what I did last week

Made enhancements on buster branch[1]
added auto-config feature
enabled autologin and guestlogin
More functionality to test.sh
Wrote accounts.sh script to manage user accounts in bulk.[2]
Created a bug report as a request to merge the edu packages that I created[3]
Provided dhcp range to dhcp server for testing[4][5]
Updated documentation[6]
Removed vagrant-cachier plugin, enabled dns[7]
We are also trying to involve the other students that wanted to work on this project[8]

Plans for next week and after

Merge request, bug report for education packages has been already made. Will remove edu.shscript as soon as it is merged.
Will update vagrant box for linux mint tara after a new release.

Pull requests Created

Request to merge education packages

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=905006

Current status of project

docker-scripts/dev--LTSP

Thats all for now folks. So that’s it for now, watch out this space for more blog posts for the whole summer.

Thanks for reading! Be sure to click claps below to recommend this article if you liked it.
You can connect with me on Github, Twitter, Linkedin :)

Week 11: Google Summer of code with Debian

Deepanshu — Sun, 29 Jul 2018 14:09:41 GMT

Here is what I did last week?

For most of the time in the week I was working on building packer template linux mint 19[1]
updated readme file[1]
Added education packages to debian edu. Improved primaryschool package[2]
Released vagrant box version 1.0.2[3]
Made some changes in virtual ltsp project to adapt to version 1.0.2 [3]
Studied how to use reportbug and filed bug (severity: wishlist) using it.
All the improvments that were made in bionic branch are now made in buster branch as well [4]

Problems I could not solve

I had to wait for bug to get reported to merge the packages [5]

Plans for next week

Improve documentation
Work on buster branch [6]
Test everything to make sure everything works as expected [7]

Pull requests created

Issues worked on

Current status of project

docker-scripts/dev--LTSP

Thats all for now folks. So that’s it for now, watch out this space for more blog posts for the whole summer.

Thanks for reading! Be sure to click claps below to recommend this article if you liked it.
You can connect with me on Github, Twitter, Linkedin :)

Week 10: Google Summer of code with Debian

Deepanshu — Sun, 22 Jul 2018 11:59:04 GMT

What I did last week?

Implemented auto_config feature in Vagrantfile[1]
Made several changes in provisioner script to adapt to the newer version of ltsp[1]
Fixed DNS issue in ltsp setup[2]
Added guest-login and auto-login functionality in ltsp server for ltsp client[3]
Improving the Debian Edu packages for uploading them to mentors.debian.net

Problems I could not solve

Still, need to make some improvements in Debian Edu packages before uploading them

Plans for next week

Upload the packages to mentor.debian.net[4]
Create scripts to automatically build vagrant boxes[5]
Create a script to manage bulk user accounts[6]
Then I will start with testing again, improve documentation to make final upload of code to google

Pull requests created

Issues worked on

Current status of project

docker-scripts/dev--LTSP

Thats all for now folks. So that’s it for now, watch out this space for more blog posts for the whole summer.

Thanks for reading! Be sure to click claps below to recommend this article if you liked it.
You can connect with me on Github, Twitter, Linkedin :)

Week 9: Google Summer of code with Debian

Deepanshu — Sun, 15 Jul 2018 12:13:23 GMT

What I did last week?

Most of the time week was devoted for testing the entire project in various distribution, cases and enviroment.
Also tried using Virtual ltsp server with 10 different clients on real LAN.
Next task to do was fix the issues found. One such issue was fixing IP address issue caused by switching to linux mint.[1]
Another issue was internet was not working on on ltsp client. Fixed that one[2]

Problems I could not solve

Didn’t get time to work on autologin and to write packer script to create vagrant box. As lot of time was given to testing
Still trying to work on auto_config of network part of vagrantfile

Plans for next week

Implement networking in Vagrantfile with auto_config[3]
More testing[4]
work on autologin feature[5]
write packer script to create vagrant box[6]

Pull requests created

Current status of project

docker-scripts/dev--LTSP

Thats all for now folks. So that’s it for now, watch out this space for more blog posts for the whole summer.

Thanks for reading! Be sure to click claps below to recommend this article if you liked it.
You can connect with me on Github, Twitter, Linkedin :)

Week 8: Google Summer of code with Debian

Deepanshu — Sat, 07 Jul 2018 21:27:55 GMT

This was interesting week. I was working on the project all the time in weekdays.

This is one of the major achievement

Dope Looking linux mint client :P

Here is what I did last week?

Implemented vagarant cachier plugin to speed up testing, installation and configuration of ltsp setup[1]
Learned how to build vagrant box from scratch blog post and wiki page for the same[2][3]
Created a vagrant box for linux mint and uploaded on vagrant cloud. Also released a second more compack version[4]
Fixed ltsp client login issue[5]
Added variable to settings.sh configuration file to add vagrant box instead of hardcoding it[6]
Updated docs[7]
Added linux mint 19 xfce ltsp box[8]
Added testing steps, commands, etc in readme file[9]

Problems I could not solve

Have to create packer scripts to automatically create vagrant boxes from iso[10]

Plans for next week

Building a LTSP server container with docker-scripts[11]
Finsh pending tasks
Implement autologin for ltsp clients[12]

Issues worked on

Pull Requests Created

Status of the project

docker-scripts/dev--LTSP

Thats all for now folks. So that’s it for now, watch out this space for more blog posts for the whole summer.

Thanks for reading! Be sure to click claps below to recommend this article if you liked it.
You can connect with me on Github, Twitter, Linkedin :)

Building Custom Vagrant box

Deepanshu — Wed, 04 Jul 2018 11:54:53 GMT

Building custom Vagrant box is not a very difficult task. But on the Internet, you would find people doing it many different ways. This creates a lot of confusion about which guide to follow. So it is good to write my own blog about it So that I can be useful for others and for me in the future.

Also, it is one of the task for my Google summer of code project Virtual LTSP Server

Before going into building vagrant box. A little bit introduction about vagrant-

Vagrant is a tool for creating virtual test environments. By just three commands you will have your vagrant box set up and running.

vagrant init precise32
vagrant up
vagrant ssh

Prerequisites

You just need 2 software

Virtualbox — If you dont have it installed you can do it by

sudo touch /etc/apt/sources.list.d/virtualbox.sources.list

echo "deb http://download.virtualbox.org/virtualbox/debian xenial  contrib" >> sudo /etc/apt/sources.list.d/virtualbox.sources.list

wget -q https://www.virtualbox.org/download/oracle_vbox_2016.asc \
    -O- | sudo apt-key add -
wget -q https://www.virtualbox.org/download/oracle_vbox.asc \
    -O- | sudo apt-key add -

sudo apt update

sudo apt install virtualbox-5.2

Vagrant — You can easily download it by sudo apt install vagrant

What are we going to do?

We are going to build a vagrant box from scratch. In the end, we will have a package.boxfile it will have all the necessary components required to start a vagrant machine.

So let’s get started

Building a box

First, we are going to create a virtual machine. For this tutorial, we are going to create a vagrant box for Linux mint Xfce. With following settings-

Virtual machine setup

Download Linux mint iso from here-https://www.linuxmint.com/download.php
Open VirtualBox click new
Set name linuxmint-19-xfce-32bit Type and version will be automatically set for you.
Set memory 1024 Mb
Virtual hard disk size 10 Gb. Hard disk file type vmdk

Then you need to go to settings of the VM

Disable audio
Disable USB ports
Enable port forwarding. Go to file > Preferences > Network

Click on the green add icon to create a new network
Then go to its settings

Then in port forwarding set this rule

[Name: SSH, Protocol: TCP, Host IP: 127.0.0.1, Host Port: 2222, Guest IP: , Guest Port: 22]

Installation

Create the virtual machine with the help of ISO we downloaded
Set vagrant user and password also vagrant

Changing root password

Open up the terminal and do a

sudo passwd root

It will prompt for the password twice. It is recommended to set the password to vagrant

Making vagrant user a Super user

Now we need to add vagrant user to sudoers list. For that, you need to edit sudo nano /etc/sudoers.d/vagrant

Add these following lines

# add vagrant user
vagrant ALL=(ALL) NOPASSWD:ALL

Press CTRL+O to save and then CTRL+X to exit in nano

Install Vagrant Insecure key

This vagrant insecure key is required for communication of Host machine to vagrant box. Don’t worry if the machine is insecure. As soon as you do a vagrant up, The vagrant will automatically replace this key with a new key pair for better security. Install this vagrant key by executing these commands on the terminal

mkdir -p /home/vagrant/.ssh

chmod 0700 /home/vagrant/.ssh

wget --no-check-certificate \
  https://raw.github.com/mitchellh/vagrant/master/keys/vagrant.pub \
  -O /home/vagrant/.ssh/authorized_keys

chmod 0600 /home/vagrant/.ssh/authorized_keys

chown -R vagrant /home/vagrant/.ssh

Update and upgrade

It is recommended to update and upgrading before packaging the box

sudo apt update --yes
sudo apt upgrade --yes

OpenSSH Server

apt install --yes openssh-server

sudo sed -i /etc/ssh/sshd_config -e \
    "/#Author*/ c AuthorizedKeysFile %h/.ssh/authorized_keys"

sudo service ssh restart

Installing Guest tools

sudo apt install --yes gcc dkms build-essential \
    linux-headers-server

These guests tools are required. If this you forget to install these the synced folders will not work. However, there is a plugin in vagrant vagrant-vbguestthat can automatically install these. This plugin can also be useful if the vagrant box by default does not come with this plugin preinstalled.

Take a look that this for example — https://github.com/fujimakishouten/vagrant-boxes/issues/1

Install VBoxLinuxAdditions

Inside VM window go to Devices > Insert Guest Edition CD image… It should open up a disk File manager. Copy the path. It should be in the format of /media/vagrant/VBox_GAs_x.x.x/. In my case it is 5.2.14. You just have to add VBoxLinuxAdditions.run after the path. So that it looks something like this-

sudo /media/vagrant/VBox_GAs_5.2.14/VBoxLinuxAdditions.run

This is the best way to this. Otherwise, you would have to find the version and LinuxAddition from here manually(not recommended).

Then you should restart the box.

Package the box

Congratulations we have almost created the vagrant box. Now we just have to create a package.box file. We can do it with the help of vagrant packagecommand. Create a linuxmint-19-xfce-32bit folder in your home directory

mkdir ~/linuxmint-19-xfce-32bit

cd ~/linuxmint-19-xfce-32bit

vagrant package --base linuxmint-19-xfce-32bit

==> mint: Attempting graceful shutdown of VM...
==> mint: Forcing shutdown of VM...
==> mint: Exporting VM...
==> mint: Compressing package to: /home/pc/linuxmint-19-xfce-32bit/package.box

Testing

Then you need to add this box and then initialize. You can do it by

vagrant box add linuxmint-19-xfce-32bit package.box
vagrant init linuxmint-19-xfce-32bit
vagrant up

Connecting to box

Now you can ssh into the box by

vagrant ssh

Now you can use this like every other vagrant box.
Congratulations! We have created a vagrant box from complete scratch.

That’s all for now folks. So that’s it for now, watch out this space for more blog posts for the whole summer.

Thanks for reading! Be sure to click claps below to recommend this article if you liked it.
You can connect with me on Github, Twitter, Linkedin :)

Week 7: Google Summer of code with Debian

Deepanshu — Sun, 01 Jul 2018 08:56:53 GMT

Here is what I did last week?

Created fully functional test.sh script for testing. It will create virtual interface, startup ltsp-server, and dhcp server and then the client with a single command and destroy them with another.[1]
Differentiated between ltsp-server and ltsp-server-standalone package by looking at package source[2]
Added GUI to vagrant box[3]

Several Improvements in `settings.sh` configuration file.[4]
Created a second troubleshooting page and testing plan. Mentor wrote a better testing plan for me [5]
Enabled debugging on all shell scripts[6]
Added edu.sh that creates Debian packages from a custom source of and installs them to ltsp server vagrant box[7]

Problems I could not solve

Working on Improving docs[8]
Tested internally and externally only, have to test on real LAN[9]

Plans for next week

Next week I am planning on testing according to testing plan and Fixing all the bugs that I find.[10]
Will be improving the docs[11]
Create custom vagrant box[12]
Explore and implement vagrant caching[13]

Pull requests Created

Issues worked on

Status of the project

docker-scripts/dev--LTSP

Thats all for now folks. So that’s it for now, watch out this space for more blog posts for the whole summer.

Thanks for reading! Be sure to click claps below to recommend this article if you liked it.
You can connect with me on Github, Twitter, Linkedin :)

Week 6: Google Summer of code with Debian

Deepanshu — Sun, 24 Jun 2018 18:49:28 GMT

What I did last week?

added test.sh script on buster branch that creates virtual interface[1]
made several improvements in `test.sh` script to create a virtual network in both buster and bionic branch[2][3][4]
Informed the maintainers about a minor bug in official ubuntu ltsp-pnp page. It is now fixed.
wrote wiki page on how to troubleshoot ltsp setup[5]
wrote a blog on how vagrantc(creator of ltsp) helped me fix the client login problem.[6]
studied about building Debian package from resources collected last week
wrote a script to automatically port and install ubuntu-edu packages to Debian(open PR and needs some improvements)[7]
Searched and tested the difference between ltsp-server and ltsp-server-standalone package[8]

Problems I could not solve

deb folder currently has unnecessary files. Need to remove those
did not get time to study how to support raspberry pi clients by ltsp server
finish up the work between differentiating both ltsp setups[8]
need to split testing wiki page into automated and manual testing pages

Plans for next week

finish up the remaining work in mentioned in problem not solved
work on issues #58 #59 #60 #61 #62 #63 #64
research on what needs to be done in ltsp-server to support RaspberryPi clients[16]
added more features to ltsp-manager[17]

Pull requests created

Wiki pages That I wrote

docker-scripts/dev--LTSP

Other blogs that I wrote this week

Solving LTSP Client Not able to login Issue

Issues to work on next week

Thats all for now folks. So that’s it for now, watch out this space for more blog posts for the whole summer.

Thanks for reading! Be sure to click claps below to recommend this article if you liked it.
You can connect with me on Github, Twitter, Linkedin :)

Solving LTSP Client Not able to login Issue

Deepanshu — Thu, 21 Jun 2018 12:03:39 GMT

Couple of weeks back I was facing a strange problem with my GSoC project virtual ltsp server. I had created vagrant provisioner scripts to automatically install ltsp in a box and boot thin clients from it. But the problem was I was not able to login. Here are the screenshots of the error

and then this

I started looking over the internet if someone else has faced the same problem in ltsp. I checked with the old messages on mailing list, popular google searches. Then I also asked some of acutal people using on ltsp on twitter(some friends that I made during community bonding period). I also checked thorugh all the chats since 2014 on the irc channel :P

I did not find any leads. I was also not even sure if this was a problem with LTSP or with ssh in vagrant. Most solutions pointed towards creating a new ssh key pair. But that is not usually what we do with vagrant. If I do that I wont be able to a `vagrant ssh` to login into the box.

Then I decided to ask people on #ltsp irc channel. I waited around 1 hour for anyone to reply. Then Vagrant Cascadian, The creator of ltsp decided to help. He also did not see this type error before. Eventhough he has been helping people with ltsp problem since 2006. But he proposed some good way on how to troubleshoot ltsp server from client. He suggest some ways that were not even present in offical ltsp-pnp troubleshoot page.
He helped me in getting a local root shell from the client.
Suggested two ways to do that-

One was to add some lines in lts.conf. One can open it by

sudo vim/var/lib/tftpboot/ltsp/amd64/lts.conf

Then added following lines in it

INIT_COMMAND_ROOT_HASH=”sed ‘s!^root:[^:]*:!root:\$6\$p2LdWE6j\$PDd1TUzGvvIkj9SE8wbw1gA/MD66tHHlStqi1.qyv860oK47UnKcafSKqGp7cbgZUPlgyPv6giCVyCSCdJt1b0:!’ -i /etc/shadow”

Then you will able to login with username as root and password qwer1234

2. Other way is also mentioned in ltsp-pnp ubuntu page. That is to add these lines in lts.conf

SCREEN_02=shell
SCREEN_07=ldm

The offical guide suggests on using CTRL+ALT+f2 to get to TTY to log in. But since we are on a virtualbox that does not work. You have to use RT_CTRL+f2.
Finally using the second way we got the root shell. Next step would be to check connectivity to the ltsp server. We were sucessfully able to ping to the server

I tried to ssh vagrant@server from the root shell

Again got the same error. I tired to increase the verbosity by using -v option But it was not helpful either.

We tried if the `/etc/ssh/known_hosts` was present or not. The next step was to look the ssh keys. So from ltsp server I did

ssh-keyscan localhost

and from ltsp client I did

 
ssh-keyscan server

Output of both looked pretty different. Both showed different set of keys. Vagrant said that is the issue and confirmed this was not an issue with ltsp but with vagrant box. And suggested to get help from the mentors.
All this was done during day and night. Finally after getting some lead I went to sleep at 6:30 am :p

Later next day I contacted the mentors. They suggest a way by using the keys provided by hashicorp. How ever it was not the recommended way as it not secure.

Then I was suggested to take a look at the logs. I did a

tail -f /var/log/syslog

and ssh from the client. It said something like client dropped the connection. Then I looked for the ssh config file.

In /etc/ssh/sshd_config password authentication yes was commented out. I removed the comment symbol and restarted ssh service with

service ssh restart

And again tried booting from the client and this time it was successfully able to login. I got this.

So probably the fix was easy. But it was kind of difficult to relate error message to the error. That why you need to learn two skills to solve any linux related problem -

Know how to increase verbosity
Know how to read log messages

With these 2 skills in hand you will be able to solve most problems in linux.

I know I did waste quite some time on this issue. But there several benefits too. Like I got know more people in ltsp community(via twitter,irc), I learned how to troubleshoot ltsp, learned to read log properly. Also the creator of ltsp know about this project they are happy to see someone working hard for this :)

Thats all for now folks. This was the first proper actual blog. Since most posts are just weekly reports. Watch out this space for more blog posts for the whole summer.

Thanks for reading! Be sure to click claps below to recommend this article if you liked it.
You can connect with me on Github, Twitter, Linkedin :)

Stories by Deepanshu on Medium

Google Summer of code at Debian Final Report

Google Summer of code 2018

Table of contents

Introduction

Final Summary

Deliverables

Virtual LTSP Server

Pull request made

Commits made

Issues worked on

Packer scripts to create vagrant box

Linux mint tara vagrant box

Ported edubuntu packages from ubuntu to debian

Documentation

Weekly reports & Blog posts

Other contributions

Thank you

Week 12: Google Summer of code with Debian

Here is what I did last week

Plans for next week and after

Pull requests Created

Request to merge education packages

Current status of project

Week 11: Google Summer of code with Debian

Here is what I did last week?

Problems I could not solve

Plans for next week

Pull requests created

Issues worked on

Current status of project

Week 10: Google Summer of code with Debian

What I did last week?

Problems I could not solve

Plans for next week

Pull requests created

Issues worked on

Current status of project

Week 9: Google Summer of code with Debian

What I did last week?

Problems I could not solve

Plans for next week

Pull requests created

Current status of project

Week 8: Google Summer of code with Debian

Here is what I did last week?

Problems I could not solve

Plans for next week

Issues worked on

Pull Requests Created

Status of the project

Building Custom Vagrant box

Prerequisites

What are we going to do?

Building a box

Virtual machine setup

Installation

Changing root password

Making vagrant user a Super user

Install Vagrant Insecure key

Update and upgrade

OpenSSH Server

Installing Guest tools

Install VBoxLinuxAdditions

Package the box

Testing

Connecting to box

Week 7: Google Summer of code with Debian

Here is what I did last week?

Problems I could not solve

Plans for next week

Pull requests Created

Issues worked on

Status of the project

Week 6: Google Summer of code with Debian

What I did last week?

Problems I could not solve

Plans for next week

Pull requests created

Wiki pages That I wrote