Let’s keep the same setup described in my last story: smartphone running the app, Fiddler as a proxy and requests analyzer, browser to view the web version.
I’m going to stick with the same target “Dr. Cannabis” / @user3k25jojtkg and head to the web view first: https://www.tiktok.com/@user3k25jojtkg
Number of videos
First, let’s find how many videos the user has posted by checking the source and the JSON response intercepted by Fiddler for the profile (/aweme/v1/user/profile/other/?):
source: "videoCount":34 & "video":34 / JSON: "dongtai_count":34 & "aweme_count":34
34 is the number of videos online.
Downloading the videos
Next, lets try to download these videos. Obviously, you cannot just “right-click” and “Save the video as…”. To grab the URL for the video, I am going to use a technique I first read about in a tweet from @osintcombine:
I will just tweak this technique so I don’t need to “view the desired video”.
Actually, hovering the mouse over the video is enough to trigger the event that will show the URL in the network tab.
I am still experiencing with that but this technique allows you to download the video without viewing and potentially changing the view counter.
Open the profile page and launch the Developer tools (press F12 or Ctrl+Shift+I). Go to the “Network” tab and enter ?a= .
You may have to press the “Filter button” to activate the filters.
If you reload the page now, you will get the URL for the first video:
To view the URL, just click the “Name” once, it will open a new pane. Select “Headers” and you have the full address that you can now copy (right-click, “Copy”, “Copy link address” will also work!):
Now that you have the URL, you can simply paste it in a new tab. Double-clicking the “Name” will open the video in a new tab too.
In this new tab, simply “right-click” on the video and “Save video as…”. Done!
Grabbing multiple URLs
For the other videos, don’t click anything, use the “Clear” button next to the “Filters” and simply hover your mouse on the videos you want to download. This will populate the “Network” tab with the URL for the next video. Then, Copy/Paste, you know the drill!
Downloading ALL!
If you really want to download all the videos, you can find the number of items you have to download (see above) and then very carefully and slowly point your mouse at each video until you reach the end. Actually, just be sure to hover on every video, we will deal with duplicates later.
When you have everything or more (there’s a counter at the bottom left of the Developer tools), right-click on the “Names”, select “Copy” and one of the “Copy all as…”. I chose cURL so I can manipulate everything as text.
If you just need the addresses, the block of text contains a lot of URLs and other useless stuff. Let’s bake it with CyberChef! The recipe is the following:
- Extract URLs
- Regular Expression / User defined / Output format= List matches:
https://.+vr= - Unique
The Output is 34 lines of data, specifically 34 lines of videos URLs for you to play with! If you want to download them in batch, check wget (for example, here). Just add a new ingredient/operation to the previous CyberChef recipe:
4. Find/Replace (Simple String)
vr=
vr= / FILE
Save the output (OUTPUTFILE.txt) as a file and then run wget:
wget -i 'OUTPUTFILE.txt'What about the app and the proxy? Well, get ready. Got to the profile and scroll down the videos. Again, you don’t need to actually view them. In Fiddler, add a URL filter for aweme/post .
The sessions response are in JSON, as usual. For each response, you will get the same information for a variable amount of videos under aweme_list . Each {} represents a video.
Back to what Fiddler intercepted. Select all the sessions you have filtered and export them as *.har (File -> Export Sessions -> Selected Sessions -> HTTPArchive v1.1). Bake the file with a slightly different recipe in CyberChef:
- Extract URLs
- Regular Expression /User defined:
http://.+vr= - Unique
You will not get 34 lines though. You will get 340!
Each video on TikTok has multiple sources: the raw video, the video presented to the users and a sound (thanks to Jake Creps and his webinar with @skopenow for the inspiration). It means that in the JSON sessions, you will get multiple versions of these three elements and you will end up with 10 URLs per video. I don’t see any other way than parsing these JSON with a script at this point, something I will probably try to create soon. Or just download all the versions… storage is cheap nowadays!
Here’s where to find the different sources in aweme_list / {}though:
Full video without “watermarks” / stickers still present:
"video": {
"play_addr": {
"url_list": [
"http://v16.muscdn.com/ea1bac1fe719a951bd3e505d34bd266f/5e9ef9f5/video/tos/useast2a/tos-useast2a-ve-0068c002/ffa528b3c36e4f1896ed303b1a1fb419/?a=1233&br=1864&bt=932&cr=3&cs=0&dr=0&ds=6&er=&l=202004210748580100991810210E10F3B3&lr=all&qs=0&rc=amp4bGRtdW41dDMzaDczM0ApaGk1aGZkNTw0NzQzMzdkNWc0MGs2NS5vaDZfLS1gMTZzc2FeX2EzMV9jYi02MTVjYi86Yw%3D%3D&vl=&vr="Full video with “watermarks”:
"video": {
"download_addr": { "url_list": [
"http://v16.muscdn.com/db345a36bb0416fde5602aa17a8b004b/5e9ef9f5/video/tos/useast2a/tos-useast2a-ve-0068c004/64a21c1bcd0f404eab19ca01a1061190/?a=1233&br=1980&bt=990&cr=3&cs=0&dr=0&ds=3&er=&l=202004210748580100991810210E10F3B3&lr=all&qs=0&rc=amp4bGRtdW41dDMzaDczM0ApPDxlNzs6ODs6N2gzOzs3M2c0MGs2NS5vaDZfLS1gMTZzcy4zMWJeYC4xYV8yMzZhMzE6Yw%3D%3D&vl=&vr="
Sound only:
"video": {
"bit_rate": [
{
"play_addr": { "url_list": [
"http://v16.muscdn.com/08f607f1b969f52439b724f6414fd751/5e9ef9f5/video/tos/useast2a/tos-useast2a-ve-0068c002/389f652387934702b2c0550881de7c77/?a=1233&br=902&bt=451&cr=3&cs=2&dr=0&ds=6&er=&l=202004210748580100991810210E10F3B3&lr=all&qs=11&rc=amp4bGRtdW41dDMzaDczM0ApNztpNGhkaTs0N2c6NjRlZGc0MGs2NS5vaDZfLS1gMTZzc2BgMF4uLS41NGMtMWMvNTM6Yw%3D%3D&vl=&vr="
MP3 (sound only):
"music": {
"play_url": {
"uri": "http://p16.muscdn.com/obj/musically-maliva-obj/1663708926267445.mp3",
"url_list": [
"http://p16.muscdn.com/obj/musically-maliva-obj/1663708926267445.mp3"Video Information
We have all the videos and we can “Sector035” them all (read: try to recreate what this great OSINTer does with images and geolocate/identify elements on them). In the meantime, let’s find more information on the metadata.
For the web view, you can now right-click the video and copy the link or directly open it in a new tab.
Open a new tab end paste the link you just copied.
This will allow you to access the actual web preview for the video, a web preview with a source page you can dissect.
On the app/proxy side, you actually have every information on each video in the JSON you already collected when browsing the videos.
Video ID
Sometimes, we need the exact ID of an object posted online. Because people from legal are a pain.
The video ID is actually the number you will find in the URL:
https://www.tiktok.com/@user3k25jojtkg/video/6814555768277175558If you really insist, you can also find it here:
source: "uniqueId":"user3k25jojtkkg","id":"681455576827717558" / JSON: aweme_id=681455576827717558
Video Date
The creation date and time are available in multiple places:
- Video page source:
"createTime":"1586637406"/ UNIX Timestamp (Use CyberChef to convert: Sat 11 April 2020 20:36:46 UTC) - App “aweme/post” JSONs
"create_time":1586637406/ UNIX Timestamp (Use CyberChef to convert) - Developer tools
Remember when we viewed the “Headers” of the video? Opening the “Headers” pane will also give you a “Last-Modified” date and time that, in my tests, seem consistent with the the moment a video was uploaded.
Video Description
The yellow box on the image is actually the video description. Not surprisingly, you can also get the text from the source of the video page, in 12 places actually! It is present in JSON as well.
Source: "desc":"#like #tiktok #love #thc #cbd #education #instaweed #cannabis #followme #joint #life"
JSON: desc=#like #tiktok #love #thc #cbd #education #instaweed #cannabis #followme #joint #life
Looking at the JSON adds some information about the description. On the video available via the app, the hashtags are actually embedded in the video. The text_extra array has extended descriptions for each word that appears in the video.
Watch out though: text can also be integrated as stickers, referenced under interaction_stickers without the actual text in any item!
JSON responses also contain a desc_language object but it is often set to “un” which probably means “unknown”.
Extended Information
The JSON responses for the videos contain a lot of information that’s not available on the web version.
Each object ({}in Fiddler) has a full “author” profile for the user, a broad choice of URLs for different versions of the sources of the video, duration, risk_info (!!), status and so on. For the moment, I will try concentrate on elements that may have an direct investigative value which is, I admit, quite subjective!
Region
The JSON includes a region for each video. I don’t know if it is only a repetition of the region for the user: region=CA
Statistics
The statistics array contains different information on the interactions with the video
"statistics": {
"aweme_id": "6814555768277175558",
"comment_count": 6,
"digg_count": 201,
"download_count": 3,
"play_count": 3530,
"share_count": 8,
"forward_count": 0,
"lose_count": 0,
"lose_comment_count": 0,
"whatsapp_share_count": 1If most of these objects are obvious, I am not sure about the download, lose_count and lose_comment_count objects yet.
Music
The music array contains advanced information about the sound used in the video. For our target, the sound was simply recorded by the user but if your are investigating someone who actually used another source, this is where you will get the title, artist or author, owner ID, owner nickname, etc…
Note: actually viewing the video on the app doesn’t show more metadata.
Views, Comments and Likes
Statistics
Viewing comment or likes is impossible on the web view without being logged in. The only information we can get without being logged in from the page source regarding interactions is limited:
"commentCount":6,"diggCount":201,"playCount":3534,"shareCount":8
The information are more detailed in the JSON statistics item (see above). I haven’t found a way to see the “diggs” in the app but the comments are easily accessible by simply touching the icon. As usual, the response intercepted by Fiddler will come as a JSON.
To filter it out from the comments sessions, you can use the following partial URL: comment/list .
The format is familiar with a comments array and each comment in another {} in Fiddler and each comment reveals more useful/interesting information:
Comment ID:cid=6817740043030462469
Unix timestamp for the comment:create_time=1587378806
Converted with CyberChef: Mon 20 April 2020 10:33:26 UTC
Comment content:text=@jakemclellan0 look at this
Full User profile with some of the arrays we already know:
"user": {
"uid": "6604404550798344197",
"nickname": "nathan pendlebury",
"gender": 0,
"signature": "Here to make friends and just for fun 👍 taken 10/09/18 🔒",
"birthday": "1900-01-01",
"unique_id": "nathanpendlebury",
"bind_phone": "",
"region": "AU", "ins_id": "",
"google_account": "",
"youtube_channel_id": "",
"youtube_channel_title": "",
"apple_account": 0, "is_phone_binded": false,
"twitter_id": "",
"twitter_name": "",
"has_email": false, "language": "en",
Comments on the web
If you login with a valid TikTok account, you get access to the comments on the TikTok webpage. Exporting them is messy as the comments are loaded as you scroll them: you have to expand the responses to comments and then you could select them, copy and paste. Another way is to access the container in which the contents are stored but again, it’s dirty:
Once the video is loaded in the new tab, you have to scroll all the comments, and expand all the replies. Then, open the Developer tools and go to the “Elements” tab. Hit Ctrl + f to find the container, designed by jsx-2690581477 comment-container .
Right-click it and choose “Copy” and then “Copy element”:
Your now have a text container with all the comments. I don’t have a good solution to nicely extract the information. I played with regular expressions to “beautify” it after pasting it in Notepad++ but nothing magical happened.
The date/time stays relative (“20h ago”, “2d ago”), for example. I really prefer the JSON method but if it is not available for any reason, this web/container technique will “work”.
Again, this story is a work in progress. Especially if TikTok changes everything each time I reload a profile. I hope this will help some investigator out there.
Writing these stories has sure forced me to abandon the comfort of the browser and to embrace the future of OSINT where apps are the new queens!
Stay safe and happy TikToking…or whatever you want to call that!
