Sharing a fake “Internet” Connection on OSX
Imagine the scenario: you need to simulate a router with working internet access and only have a Macbook Pro with its built in Wifi card. How are you gonna’ Macgyver your way out of this one? What’s that? That’s a silly scenario? Well suppose you…well I mean you might want to…yeah it’s a bit silly.
But it turns out it’s not a wholly useless functionality. I recently found myself in a position where I wanted to capture the outbound traffic from an application on my iPhone. It seemed straight forward enough: create an ad hoc network from my Macbook Pro, connect my iPhone to it, and then just sniff the traffic using Wireshark. However this turned out to be a bit more of a pain than I initially planned for.
Now OSX has two ways of creating an ad hoc network that I’m aware of. You can click on the WiFi icon in the menu bar and then hit “Create Network…”.
This kind of does what we’re ultimately attempting to do; it creates a new WiFi network managed by your Mac that you can connect your iPhone to. While this is great if you want to do something like share documents or a try out a small LAN, it is inherently cut off from the Internet and OSX knows this. Furthermore, a device like an iPhone realizes this as well and will throw up a big warning flag when you connect to this network and, from what I could tell, won’t even bother attempting to send data out to the Internet.
Your other option is to go to “System Preferences” => “Sharing” => “Internet Sharing” and share an Internet connection from another interface over your Wifi signal (or share WiFi over Ethernet or whatever you want to do).
This will also create a new WiFi network that your iPhone can connect to, and this time it will treat it as if it were connected to any other router or access point. This is more what we’re looking for, and this is the path we’re going to go down.
The problem is that you need another interface besides your built in WiFi card, which I’ve so arbitrarily limited you to in the beginning of this article, to share from. Sure you could go buy USB WiFi adapter, but those are like what? 10 bucks? But hey that’s fine, you’ll just go to “Network Preferences”, create some dummy interface like a “Thunderbolt Bridge” or VPN and use that. Well that’s a great idea until you realize how moronic it is.
See, because while you might have, at best, an elementary understanding of networking, probably due to wasting many of your teenage years and dropping out of the liberal arts program at two (count them, two) colleges, OSX knows better and won’t let you share this kind of network. But why is that? Well I guess you were just trying to keep yourself distracted from the pressures of high school and later felt you could learn more from work experience than from higher education. I guess you have a lot of thinking to do, maybe your father was right.
The point is, obviously you can’t share an internet connection from a device that’s not connected to anything; and while OSX is too friendly to throw any semblance of an error saying something helpful like “Hey, dope, I can’t share that connection, it’s not real”, it also won’t actually create the new ad hoc network either. Instead it will just smile and nod complacently and hope we all forget this embarrassing moment happened.
So we need to create a virtual interface and we need to have it actually be responsive. What we need is a TUN or TAP interface. TUN and TAP devices, incase you’ve never run into them before, are very simply virtual devices that are accessible in user-space. They appear to the Operating System like any other physical device, but it’s entirely manageable through software rather than a hardware backed network adapter.
But here we run into the issue that you always seem to run into with Apple when you try to do anything fun. See, mamma OSX knows what’s best for you and she doesn’t want you running around with virtual devices, you might scrape a knee and hurt yourself. No, you just stay here by the standard WiFi, play with your Photoshop and get yourself something nice off of the Apple Store; Hah, “I want to set up a TAP device”, are you trying to kill your poor mother?
Luckily, if you’re willing to devote the time, there are ways around Apple’s hand holding. First things first, you’ll need to download and install the excellent TunTap (apparently also available via Homebrew; this adds the actual kernel support you need for a Tun/Tap interface and makes it available to you as a number of special files (character devices to be specific) at /dev/tun[0–9] and /dev/tap[0–9]. At this point, if you’re new to TUN and TAP devices, you might be wondering what the difference is between the two. In order to give a satisfactory explanation I feel I might have to first give a layman’s explanation of the OSI model. So if you’ll excuse the digression…
Let’s think of a network. It’s a pretty hectic concept, isn’t it? Data transfer is the mad rush of electric pulses down a wire, the hurried electromagnetic waves made by the beating of the WiFi drum. In a sense, all communication is frenzied distortion. We distort our electrical signals: POWER ON, POWER OFF, POWER OFF, POWER OFF, POWER ON, POWER OFF, OFF, ON, OFF. We distort our radio waves: MODULATE THIS SINE WAVE, CHANGE IT’S PHASE, CHANGE IT BACK. Each machine dumping out millions upon millions of bits of noise, sending it coursing through cables, sending it coursing around our bodies through the fabric of empty space. And as it dumps this chaos into the ether its absorbing it all back in. It listens to those pulses, those modulations, the distortions which signify information. Listening, blindly and alone, to the ferocious noise and the relentless bedlam. Hoping to make sense of it all, to find relevance and meaning in what can only be perceived as unprovoked randomness. It’s hard not to sympathize.
But if you know what to look for, it’s not random at all. Those radio waves aren’t the mindless distortions of an erratic universe, they’re the beautiful, deliberate interpretive dances of data, choreographed down to the sub nanosecond. That incessant pulsing of electricity isn’t hateful ramming, it’s the notes of a song, carefully measured at just the right tempo so you can follow along.
How do we do this? How do we make sense out of senseless noise? Well with protocols. We all agree before hand how we plan on sending our data and that way we all know what we should be listening for. And these protocols aren’t some kind of fundamental laws of nature; they’re just made up! So don’t get intimidated by them, they ain’t all that.
One suggestion for handling data that was developed back in the 1980s was the OSI model. The concept itself is simple, abstract data communication down into seven layers, and keep relevant information confined to these separate contexts. Let’s think of communication as a construction site where plumbing’s getting installed.
- Layer One is the physical component of the communication. He’s the guy on the ground who will actually be laying the pipes. He doesn’t care about what the building is, what it does, how the water will be used. That’s something the eggheads at higher levels can waste time on, he needs to know specification and materials. He’s pragmatic because he has to be, he’s responsible for what’s physically going on behind the scenes and is what makes things actually work. He’s the workhorse. “Whatta’ we got here? WiFi job? What are we looking at, 2.4 GHz or 5 GHz? What do you mean you don’t know? Howdya’ expect this thing’s gonna work if we don’t know what material we’re working with?”
- Layer Two is the data link layer. He’s the foreman. He’s down there in the field and keeping track of everything, making sure all the pipes are going to the right places. He needs to have a higher view of everything but knows his way around with a hammer too and will sometimes step in if he see’s something wrong. “Alright, we’re gonna need a pipe going from MAC 2C:CC:39:38:3C:BB to MAC 9C:88:07:9B:44:18. Hey, Conti, you’re dropping bits all over the place! Where’d you learn signal timing? Move over, I got it, you want something done right you gotta do it yourself.”
- Layer Three is the network layer. Think of him as the project manager. He knows the layout of the system backwards and forwards and exactly how he wants everything to look. While the layer two is concerned with individual pipes, layer three is looking at the system as a whole and that all the different parts of the building are connected in one way or another. “I just got off the phone with 172.16.0.1 and he says we can connect to his pipes so we can get linked up with 18.104.22.168”
- Layers 4–7 take a bit of a conceptual turn. At these levels we’re assuming that all layers 1–3 did their jobs and now we can focus on what we’re actually sending. This is where you see things like UDP and TCP (layer 4) and HTTP and FTP (layer 7) built on top of them.
One thing you might notice about this classification system is that it barely holds up under a faulty metaphor for a construction site, let alone to the entire spectrum of human communication. It’s not a science and it’s not particularly dogmatic, so it’s perfectly fine if you feel that the lines between the layers gets blurred. For instance take the ARP protocol, which we’ll get into more later. It’s traditionally considered a layer 2 protocol with it’s handling of MAC addresses but it also contains references to IP addresses which are decidedly layer 3 constructs. So where does it fall? Well somewhere in the middle, and that’s fine.
So to get back on topic; TUN and TAP. Umm, TUN is for layer 3 stuff and TAP is for layer 2. I’ll be using TAP. Now let’s move on.
So we’ve got support for TUN and TAP interfaces installed, we should be pretty much done, right? I mean how can I possibly drag this article out any longer? Well you’re forgetting that this is about OSX, you didn’t think it would be that easy did you? See, OSX is pretty weird about what it shows you and what is actually going on. For instance, go ahead and download openvpn and connect to a VPN. If you run ifconfig you’ll see the new interface is there and you can use it just as you can any other interface. But now try looking for it in the Network panel under “System Preferences”, try to find it as an option under “Internet Sharing.” Don’t see it? Of course not! You didn’t create that interface the way Apple wanted you to so you’re dead to Apple.
Luckily there’s StackOverflow, and there’s a great post there that goes over exactly what we want to do: http://stackoverflow.com/a/6375307/3289001. Essentially, we need to manually add an entry to /Library/Preferences/SystemConfiguration/preferences.plist for our tap device and associate it to our current “Location” (Purely an OSX construct for categorizing interfaces). I recommend going through the post thoroughly but for reference my entry ended up looking like this for the new device:
Now when you reboot your computer you’ll see we have a new ethernet device listed under “System Preferences” => Network.
Alright well thanks for reading, on my next blog post I think I’ll go over-What’s that? We’re not done yet? Not by a long shot. See while we’ve successfully created a new ethernet device, it’s not connected. And as we’ve already gone over, that means it’s useless to us. But that’s why TunTap is so great, it’s surprisingly simple to get that device “connected”. All you need to do is open one of those special files it made accessible to us under /dev/. So let’s go ahead and do that:
# We're essentially logging in as root so, for the love of god,
# be careful
# Open /dev/tap0 with the, completely arbitrary, file descriptor 4
Now we look back at Network preferences and…
Well what more can there possibly be to do now? If we go back to Internet Sharing, we can set it up so it shares our Tap connection over WiFi, we enabled it and of course it doesn’t god damn freaking work.
Ugh, what’s the problem now? Okay, let’s sniff out what’s going on with our devices using Wireshark. Our tap interface should show right up under “Capture” in Wireshark,
we double click and then Wireshark will start tracking all the network activity going through that interface. Except it’s totally blank. There’s no network traffic? OSX has to be sending something to our tap interface, it will at least attempt to share it’s connection, right? Well, for the first time I won’t blame Apple for this, I just blatantly misled you to more accurately take you down my process. See traffic is in fact getting pumped into our tap interface, Wireshark is just not picking up on it because it can only sniff throughput, which is effectively zero because while we opened up /dev/tap0 and while OSX is sending data to it, we’re not actually reading any of it. It’s just sitting there in limbo. So let’s go ahead and get the data flowing so Wireshark can start actually monitoring what we’re doing. In the same terminal you opened /dev/tap0:
# Read from file descriptor 4 and output everything to /dev/null
cat <&4 > /dev/null
And just like that the pipe is unclogged and Wireshark will fill up with…a lot of ARP requests.
Now, again, if you’re new to looking at network traffic you might be thinking “What’s ARP do again.” Well how do I best describe this? Hey! I know! How about a tangent?
Think of ARP like a personal ad in the paper. You’re new to town, you don’t know anybody and your address book is empty and you’d just really like someone to talk to. You know how it is, you’re not looking for a huge commitment just a nice down-to-earth device you can share things with. Somebody to give a quick ICMP call to every once in a while and see where things lead to. I bet her name is something exotic, something like 10.10.10.1. Where in this big cruel network will you find 10.10.10.1?
Luckily all is not lost because you do know one address, trusty old MAC address FF:FF:FF:FF:FF:FF. Hell, it’s pretty easy to remember. You know all you have to do is send a message there and it will make sure everyone on the network sees it. So you write up something like:
“Single easy-going network interface ISO similar. Must have IP 10.10.10.1 and be D&D free”
You put it in an envelop, stamp it with your address, “5a:75:c7:03:d5:c8”, and mail it out to FF:FF:FF:FF:FF:FF. From there it gets copied and broadcast all over the network. To abrubtly change metaphors; imagine FF:FF:FF:FF:FF:FF as a radio station everybody on the network listens to.
“This song goes out to that cat over at 5a:75:c7:03:d5:c8, looking for 10.10.10.1” (It’s my fantasy and in my mind it’s the radio station from The Warriors)
Now everybody’s gotten the message, 5a:75:c7:03:d5:c8 is looking for somebody with the IP address 10.10.10.1. The majority of people don’t care, they’ll just say “Hey I’m not 10.10.10.1” and forget the message ever existed. But somewhere out there is 10.10.10.1, and she’s listening to the radio (or reading the personal ads, or doing whatever the contrived example I locked myself into was). She’ll giggle and blush and quickly write up a letter of her own and send it straight to 5a:75:c7:03:d5:c8 saying something like “Hey heard you were looking for me, call me some time at DE:39:19:8E:88:32 ;)”. And now the two love birds can have their own intimate chats directly to one another. If this doesn’t give a totally clear description of ARP I’m sure it at least shows why I shouldn’t date.
So looking at the Wireshark frames, we can see that tap0 is getting pelted with a bunch of broadcast requests following the ARP protocol asking everyone on the network for the MAC addresses of a few IPs. Hmmm, how should we handle this? Well let’s close the tap device and have a think about it.
# Close file descriptor 4
So we have all these ARP requests coming in and we need some way to placate them well enough for OSX to think tap0 isn’t just some dummy interface we pulled out of our ass (which it is). What do you think? I’m sure there’s programs out there that would let us hijack network traffic and inject our own and then we could-wait, you’re overthinking this. We’re not using a “real” interface where we need to work around physical hardware and the kernel. We’re using TAP, which was made for this sort of thing! And while now we’re blindly just reading from /dev/tap0 and dumping to /dev/null to quickly track the traffic with Wireshark, there’s nothing stoping us from actually using /dev/tap0 as what it actually is, a file. And as for parsing the data we read off of /dev/tap0 and crafting a response, I’ve got the perfect tool.
If you’ve never used Scapy before, you’re really missing out. It’s a really simple and useful python utility for packet manipulation. It’s a great library to have in your toolbox. Installing it is a bit more involved than just running a “pip install” but still very straightforward (Note that if you want to install Scapy into a virtualenvironment, you only have to run the “python setup.py install” steps with the virtualenv activated). I’ll now turn it over to python code I eventually wrote with ample comments.
Now if you run this python script under sudo:
sudo python tap_interface.py
The interface will get created, OSX’s ARP requests should be getting suitably responded to,
It should be noted that you should probably only have to run this maybe once. OSX is smart enough to realize that ARPing is a bit of a nuisance and will cache the MAC addresses it discovers. If you want to clear this cache out, all you need to do is run
sudo arp -a -d
And best of all, now if you go to Internet Sharing and activate it so your phony Tap “connection” is shared through WiFi, it will actually work!
And not only that, when you connect your iPhone to this new WiFi signal, it won’t complain about there being no internet. You can go to a browser and try hitting an website, and if you’re tracking everything with Wireshark you’ll see it…complains about DNS responses now…
But that’s fine! Something wasn’t working and now it is! And the point of all of this is that it wasn’t all that hard. I mean what? A few installations and some python code? That’s the thing. None of this is terribly hard. A pain in the ass sometimes, sure. But hard? Not particularly. Esoteric maybe, but only because it takes interest in the subject to get caught up.
So now we investigate the DNS protocol, we read up on what goes into a packet, we analyze what traffic looks like on a working network and emulate it on our broken network. And then we do it for ICMP, and DNS-PCP, DHCP, IGMP, Something-Something-P. We read up on TCP and how to send HTTP. We tackle each one at a time, keeping in mind that there’s nothing mystical going on with any of this stuff. There’s nothing here that can’t be learned, and maybe we’ll be a little smarter for having gone through the process.
Although really I should have just bought that second WiFi adapter.