Things I Learned Building the Skynet PART 2 — How to Eat Dust
We went to one of the most remote yet widely populated areas in the world to setup a very high altitude solar powered mesh network community internet in a town where temperatures range from +40'C to -40'C. Here what is happened .
LOCATION: Leh, Ladakh @ 13,500 ft
OBJECTIVE: Setup Skynet
TIME: July 2016
Skynet is a technology intervention initiative focused on developing open source technology to support the community internet movement. Free access to community internet increases equality, by reducing obstacles to critical economic, educational and health information.
The current Skynet implementation is based on ideas and resources by:
- The School Server community (http://schoolserver.org)
- VillageTelco (https://villagetelco.org/)
- Anish Mangal & David Li
It’s important to understand that all the technicalities are far easier than they initially seem. This is true especially for those with no previous experience with electrical work who may feel slightly overwhelmed initially. There is some wiring involved, yes, but it’s very simple wiring. When I was 8 years old I made a dim switch for a small light connected to the circuit. That project was far more involving in terms of wiring.
The most important aspect seems to be preparation. In my case I had Anish providing me with extensive support during the period of time I was preparing for my first deployment trip. In my case this took place over a period of 6 weeks before departure.
This post is a second part in a series, read the first part for background if you didn’t do it already >
We went to one of the most remote yet widely populated areas in the world to setup a very high altitude solar powered…medium.com
Guide to Eating Dust
In the old times they had a saying about Leh; if you are coming there, you have to either really love something there, or hate something there. Tucked in between some of the great mountain ranges of the Himalayas, Ladakh sits in high altitude on Tibetan plateau between Pakistan and China, in the state of Jammu & Kashmir in India. The only major town and capital of Ladakh, Leh, is at the altitude of 3,500 meters (10,500ft) and everything else in Ladakh is up from there. It is a bone dry desert where summer temperatures can reach +40'C while night temperatures go as low as -40'C during the winter. Few of the world’s highest motor roads and highway passes are leading to and from Leh. Including Khardung-la, the world’s highest motorable pass at around 6,000 meters (18,000ft).
There are few ways to come to Leh, of which the most adventurous is the road from Manali. You can also come by road from Srinagar, which is an amazing ride but be warned that Srinagar is nothings short of a war-zone. Curfews are not uncommon in Srinagar. It is far more militarized than for example the Hezbollah controlled southern border of Lebanon. Once you get out of Srinagar, it’s amazing. It’s also possible to fly to Leh. Altitude sickness and acclimatization wise, it’s much better to come by road than to come by flight.
Even the flight to Leh is in some sense extraordinary. After spending nearly an hour flying across the peaks of Himalaya, because there are mountains everywhere, the pilot has to turn sharply (90 degrees!) just moments before the plane touches the tarmac. Evidently only the best pilots are allowed to fly Leh routes.
The flights arrive early in the morning, and the best thing to do is to relax and take rest at least for the first day. It seems to be equally important to not fall asleep before evening on that first day. One of the most common symptoms experienced in altitude is restless sleep. Sleeping during the day doesn’t help make the night’s sleep less restless, don’t do it regardless of how tired you feel on the first day.
Countless Indian tourist go shopping instead, and many end up with oxygen tank or worse in a hospital bed. According to altitude medicine, 90% experience some symptoms in Leh altitude. Usually first few mornings my nose is bleeding, and at least on the first morning I feel like I have a hangover. If you’re in high altitude and you have not drink alcohol but feel like you are hungover, you are likely to have Acute Mountain Sickness (AMS). Feels terrible, but usually nothing to worry about if you take the right precautions such as taking more rest, drinking plenty of water and eating carb rich nutritious food. Don’t smoke cigarettes, drink alcohol or use drugs. Avoid taking medicine.
Ladakhi drink a lot of tea. Sometimes two cups at one time; one of regular Indian variety (chai) and one of the local variety (butter tea). They also love green tea, and really any kind of tea. The local tea is very strong, and leaves are boiled up to two hours before mixing with butter and milk. It’s good for keeping the body’s membrane’s more moist in the near 0% humidity, but drinking too much might not be a good idea because of the strong stimulating effect butter tea has.
Many Ladakhi are very kind and gentle. They live based on Buddhist practice, and view the world according to the principles developed in Nalanda University 2,000 years ago. “Uma”, or the middle way explains that while things appear to exist from their own side, they actually only exist in dependence to other things. Most Ladakhi are calm and centered, and are always ready to help others. Most people wear traditional clothes, recite mantras and start their day by performing offerings and by reciting one or more memorized meditation manuals referred to as “sadhana”. ‘Sadhana’ is a sanskrit word that means literally “a disciplined and dedicated practice”.
Things are changing fast like everywhere in India, but it feels that just 30 minutes outside of Leh people live like they did a thousand years ago. Much of Ladakh still do not have hospitals, and many valleys do not have TV, mobile phone, or regular RADIO signal.
In 2015 coming through the Manali road, our car was stuck in 4,500 meters (13,500ft) for 3 nights without access to other food than instant noodles. After the first night, early in the morning together with local villagers, I traversed a mountain wall and walked for 3 hours to get out from there. Such experience is not extraordinary for those traveling in the region.
Sometimes roads collapse in Ladakh, and when they do, there is no knowing of when those roads can be used again. Even though the Indian Military and BRO (Border Road Organization) have done an amazing job in building roads and other infrastructure in the Ladakh region, the landscape and conditions are simply too challenging.
I knew that I was going to experience many obstacles, and that I’d feel sick, and that I actually had no idea if what Anish had told me was actually going to work in technical terms. When your antennas are made from scrap metal tin cans, there is an element of erosion to your confidence. It seemed that it could work, and Anish seemed like someone who really knew what he was talking about.
We would not be able to control things like weather, altitude, road conditions (or dust!). So preparing for the specific needs (tools, parts, etc) of the Skynet installation would be a key.
When I went back to Europe from Rakkar Village (read part I), I became busy for about two weeks and end of May realized that unless I started to prepare, I would not be ready by 16th of June (the date of my ticket to India). I ended up doing the last content downloads for the school-server on the morning of 16th, leaving my last stable internet before the 1 month trip ahead.
I guarantee that you can not imagine how bad internet connectivity in Ladakh is, even in the center of Leh town. It is far worse than for example having a 14 kilobit modem 20 years ago in Scandinavia. Sometimes there is no connection for days, and when you have a connection, it’s usually not stable. Download speeds would be typically be a few kilobyte per second at best.
To be able to deploy community internet installations, it’s important to have a good contact in the local community. It’s better to have many if possible, as they all will have different local contacts and their own view to local affairs. In any setting resembling “rural”, understanding local affairs is a key success factor. Also, bare in mind that it will impossible to have any useful understanding of what is relevant for the local community without being appropriately connected with it.
It’s almost as important to have a good contact in the community internet community. One that has already done something similar to what you want to do. Someone like Anish. Try to have that someone “commit” to being your support person before your first deployment and join you for the actual field trip if possible. Without the support for at least pre-trip, you will feel intimidated and frustrated more than is needed. If you don’t know anyone, get in touch out of the cold. Leave a comment in below and I’ll do my best to connect you with the right people.
Most are doing this 100% at own expense in order to help others.
Train hard, win easy.
You have to make sure that you’ve done what you are planning to do before you depart for your trip. For example, in the case of xsce (“the school server”), there is up to 2 terabytes of content downloading involved, which takes time even with stable modern connection. There are countless hoops you have to go through in terms of configuring everything before the system is ready for deployment. This is true even if you start from an image provided by the community.
Access to “limited internet” is most valuable in areas where you will not have access to formal internet, or have very poor access.
If you have multiple servers (for multiple deployments), then first take one to “ready to deploy” state, take an image of that and test the image on one server. Make sure that the hard disk you’re using for the image copy process is at least same size or larger than the one you’re making the image out of. Then you’ll be able to do in a few hours of automated process what initially can easily take a week with periods of intensive troubleshooting and moderate levels of headache.
It’s a good idea to take the image even if you just had one server, as it will allow you to easily to come back to the clean initial system in case you needed to do that for any reason. Also did I mention…
>> DO IT ALL WELL AHEAD OF THE TRIP <<
You don’t want to rush when you’re learning this. Once you’ve learn the fundamentals, it’s actually very easy to go through the actual deployment. I started way too late and I can’t characterize the process as easy.
On location you want to know things like electrical and hardware shop locations, get access to rooftops and other high points, contacts withing administration offices of schools, etc. Also it’s good to think about training at least a few locals to do very simple maintenance tasks and for acting as a point of contact on the ground when you are not there. Try to make sure that you can have remote access to the network even occasionally.
Don’t count on having access to internet. Access to “limited internet” is most valuable in areas where you will not have access to formal internet, or have very poor access. So plan for that. Don’t even count on making phone calls when you need to.
It’s better to bring your own tools. At least:
- screwdriver with multiple heads
- small electrical screwdriver with very thin blade
- wire cutter / wire tool
- pliers with thin heads
- carpet knife
- electrical tape
- silicon sealant
- soldering iron
- sponge (for solder)
- good quality electrical wire
- soldering iron
When it comes to the components (routers, panels, etc) you will need, source it all well ahead of time and carry it with you. If you’re going to fly, have it in your carry-on so if your luggage is lost, you will not lose all your gear. It’s easy to get books and clothes or whatever else, but electrical components, routers, etc. will not be easy to get.
Working in remote locations
When I arrived in Leh and started to dig through everything that Anish had sent ahead of coming and that I had brought myself. It was clear that some very basic things were missing:
- usb power cables
- more dc-dc modules
- silicone sealant
- spike (to drill hole to plastic)
- toolbox (empty)
I found something resembling tupperware after visiting 4 shops, with very limited selection in size and shape. I found 3 computer shops in total, none of which sell antennas. After having visited 10 electrical shops, I did not find dc-dc modules, but one shop was carrying a consumer dc-dc converter. USB power cables are easy to come by, but quality is a real issue. You might have to test a few cables before finding one that works to start with.
By the end of the trip, we did not manage to find silicon sealant, toolbox nor a few other generally easy-to-find items. After visiting more than 10 household item shops, I found one that sold suitable “tupperware” for water proofing the routers that would all be deployed outside. When I went back the next day to pick up more of them, there was a hole in front of the closed shop.
As the first few days turned in to the first two weeks, Anish’s arrival came closer and I was able to make good progress with preparation. Everything seemed to be in place, but would we be able to successfully deploy the three separate installations we had planned?
Most of my worries were related with the Leh installation. There was a crazy amount of wifi signal in comparison to how poor connectivity is in Leh. It seemed like every house and business had their own wifi. Also we would have to transmit the signal in one instance over a kilometer through the busiest part of Leh. We thought it was very possible that nothing like this had been done before using the technologies we were going to use. Frankly speaking, I was not sure if it could be done.
[to be continued]