Tools Ways to rip entire TikTok profile?

neolith

egirl shill
Mar 11, 2022
597
37,744
1,793
0fya082315al84db03fa9bf467e3.png
if you're looking for a scriptable or automated way, yt-dlp supports entire profiles without watermark.

i just run this from my terminal. obv change the username, and the path. keep the % sections if you don't recognize youtube-dl.

Code:
Please, Log in or Register to view codes content!
 
Last edited:

neolith

egirl shill
Mar 11, 2022
597
37,744
1,793
0fya082315al84db03fa9bf467e3.png
Just consolidating my solutions into a single post and I will update it for future changes.

To attempt to rip entire profiles (updated 2023-12-14):
yt-dlp has been broken for scraping entire profiles for a bit, but someone made a working yt-dlp plugin.

1. make sure python 3 is installed and in your path (its a checkbox you need to select during installation)
2. then enter this command in your cmd or terminal.
Code:
Please, Log in or Register to view codes content!
3. then close any terminals/cmd prompts you have open, and then reopen them to update your paths/profile.
4. then run this command in either your cmd or terminal
Code:
Please, Log in or Register to view codes content!

To manually grab links and THEN download:
1. Get Tampermonkey
2. Add
Please, Log in or Register to see links and images
.

JavaScript:
Please, Log in or Register to view codes content!

3. Go to a tiktok profile and press ctrl+alt+a
4. Once it scrolls back to the top all of the links should be in your clipboard.
5. Create a text file, save it somewhere you remember and paste the links inside.
6. Make sure Python 3 is installed and in your path (its a checkbox you need to select during installation) (avoid the microsoft store version)
7. Enter the following line in your command prompt / terminal.
Code:
Please, Log in or Register to view codes content!
8. Be sure to alter the paths. And then press enter!

Changes:
  • Took a long time, but finally got the most optimal infinite scroll in js I could for tiktok that only took me ~60 seconds to load 1000 videos.
  • Updated the script to ONLY run on "
    Please, Log in or Register to see links and images
    " urls.
  • Sped up the crawl a little more.
  • Added a hotkey (alt + a) to manually stop scrolling if you just want to catch the links as is.
  • Completely moved to a userscript base.
  • Fixed the slippery auto scroll that would happen if we jiggled too soon.

Upcoming:
  • Working on a solution to input a list of users, and then have the browser crawl every profile entirely and give you all of the video links when complete.
  • Once that is completed MAYBE turn this whole thing into a wonky unpacked extension.
 
Last edited:

neolith

egirl shill
Mar 11, 2022
597
37,744
1,793
0fya082315al84db03fa9bf467e3.png
Since yt-dlp is still broken for crawling user pages, but still works ok for downloading direct video links, I made a bookmarklet for getting them all copied to your clipboard quicker than using other extensions, and typing filters, etc.

JavaScript:
Please, Log in or Register to view codes content!

and then just throw that code into
Please, Log in or Register to see links and images
to make it a clickable function.

so just load up a tiktok profile, scroll to the bottom, click the bookmarklet, then once you have your links, place them into a text file and run this from your terminal

Code:
Please, Log in or Register to view codes content!

if anyone who's proficient at js would add the scrolling to bottom step to it, please do so. I couldn't get a function for that to work right, since I'm a noob at js, and pretty much using copilot to write this for me, since all i know is python.
 
Last edited:

neolith

egirl shill
Mar 11, 2022
597
37,744
1,793
0fya082315al84db03fa9bf467e3.png
%(uploader)s puts the videos in a folder named after the user you ripped from. %(id)s gives the filename the video id, and %(ext)s gives it the proper file extension. .mp4 generally. but it's kind of useless atm since tiktok changed some stuff so yt-dlp is incapable of scraping entire tiktok accounts at the moment. but direct tiktok links do work.

so what i do at the moment is i have the Link Gopher extension installed, then i scroll to the bottom of the tiktok profile, open link gopher and choose "extract links by filter" and then i put /video/, then i copy all of those links, and put them in a file named urls.txt, and then i open a command prompt and run

yt-dlp -a urls.txt --convert-thumbnails jpg -o "%(uploader)s/%(id)s.%(ext)s" --add-metadata --embed-thumbnail
 

Obicase

Bathwater Drinker
Mar 16, 2022
61
1,732
1,252
0fya082315al84db03fa9bf467e3.png
Yo guys, sharing more info on tiktok scraping that's more or less related to the thread.

Please, Log in or Register to see links and images
is a site with its own tiktok API that archives tiktoks, including toks that have been deleted. Pretty cool to download accounts that are banned/deleted or even singular deleted toks. Note that they will only have deleted tiktoks archived IF it was archived when it was still up, if it wasn't archived before it was deleted, it's gone for good. Anyway, this site has been around for years now so that's not new.

However, what is new (to me anyway) is that you can download all of the tiktoks of an account archived in tik.fail quickly in one download.
Typing this in your url bar will create and download an archive of all the toks archived on tik.fail of that account in max resolution (replace "<username>" with the actual username) :
https://api.tik.fail/zip/<username>?hd=true

You can also replace "true" with "false" if you want lower quality only (don't do that).

You can also go on the tik.fail disc 0rd and use bot commands in a channel there to request that tikfail automatically monitors and archives accounts of your choosing. You can request monitoring on up to 10 accounts.
I can't post a link to it per forum rules, but you can find it on the tik.fail website.
 

Obicase

Bathwater Drinker
Mar 16, 2022
61
1,732
1,252
0fya082315al84db03fa9bf467e3.png
Hi, I am sharing another python script made by an Anon that kindly authorized me to share it here. I wasn't sure if he wanted his mega link posted somewhere else so I reuploded it here :
Please, Log in or Register to see links and images

This is a command line program. Pretty straight foward to use if you have already used similar scripts before (like ytp-dl that was posted in this thread). Still, if someone needs it I can make a step by step guide.
It is open source so feel free to check the code. I've been using it for a couple weeks now, no problem.

It also needs ffprobe, python, and the "requests" library. Pretty easy, more info in the README.txt or Requirements.txt.

This script cannot download stories, image posts, or private accounts (you can't give it your cookies).

However, besides those three missing features, it is still really powerful. The command at its simplest looks like this :
python awemer.py -u not.papadonks
This will download all of @not.papadonks 's videos in the highest resolution possible, in a folder named not.papadonks in the directory where the program is (if the folder doesn't exist, it will create it). The default filename for videos is VIDEO_ID.mp4, or VIDEO_ID~sd.mp4 if the video is low res (576x1024 or 1024x576).

Alternatively, you can feed it a list of usernames in a text file.
python awemer.py -u profiles.txt
Will download all of the videos of all of the profiles in profiles.txt (one username per line), each in their respective folder.

You can also use user ID instead of usernames, ie : python awemer.py -u 6998772783719285766

There are several parameters that you can add.
For exemple, -t to download all of the videos released after a certain date. For exemple, this will download every video released after the 15th of october 2023.
python awemer.py -u not.papadonks -t 2023-10-15

-u to change the directory, -p to use http proxy, -s to add video description to the filenames, -i to print user ID and secID, -ad to add the date to the filenames, -se to skip existing SD videos, -l to loop.
For more details on each parameter, check the readme in the mega link.

You can also use the executable instead of python. The command line then looks like this :
awemer.exe -u not.papadonks

I personally use this :
python awemer.py -u profiles.txt -t [yesterday's date]
"profiles.txt" contains all of the profiles that I follow (around 700), and I set the date to yesterday's date. I run this two or three times a day. This way, this downloads automatically every video released by all 700 accounts in the list, without me having to do anything.

Note that the API limit is 5000 requests per day. This is the case for all tiktok scrapers, not just this one. This is why there is an option for proxies. Like I said, I update 700 accounts per day, three times a day, and I don't hit the limit. The only time you might hit it is if you do a scrape of a large number of accounts with no -t parameter.
 

richard2480

Fan
Jun 3, 2022
20
50
97
0fya082315al84db03fa9bf467e3.png
updated my scraper. uploaded to paste.ec:
Please, Log in or Register to see links and images


decryption key (add to URL after a #) is:
dEaDmCPXW8P7bAm+qPogMq8Fs7kgpy8wdcFFi3z+tBq

inputs:
- MAX_RETRIES (for playwright)
- PRINT_OUTPUTS (obvious)
- RETRY_FAILED (retry failed downloads from previous runs)
- HOME_DIR (where to save files relative to .py file)
- USER_NAME_LIST_FILE (text file with a user name on each line)
- SORT_USER_NAME_LIST (obvious)

python packages:
Please, Log in or Register to see links and images

Please, Log in or Register to see links and images

Please, Log in or Register to see links and images

Please, Log in or Register to see links and images

Please, Log in or Register to see links and images
 

neolith

egirl shill
Mar 11, 2022
597
37,744
1,793
0fya082315al84db03fa9bf467e3.png
Please, Log in or Register to view quotes
yt-dlp has been broken for scraping entire profiles for a bit, but someone forked it and fixed it.

1. make sure python 3 is installed and in your path (its a checkbox you need to select during installation)
2. then enter this command in your cmd or terminal.
Code:
Please, Log in or Register to view codes content!
3. then close any terminals/cmd prompts you have open, and then reopen them to update your paths/profile.
4. then run this command in either your cmd or terminal
Code:
Please, Log in or Register to view codes content!
5. then run this command in either your cmd or terminal
Code:
Please, Log in or Register to view codes content!

updated on 2022-11-14
 
Last edited:

Obicase

Bathwater Drinker
Mar 16, 2022
61
1,732
1,252
0fya082315al84db03fa9bf467e3.png
Some more info regarding usernames, account ID, SecUid, and how to "track" accounts that change their usernames.

If an account changes its username and you're trying to find what the new username is, there are a few methods :

- Search for the old account name in the tiktok search bar. If the account was tagged in someone's else tok or if one of its toks was dueted or stitched, they will show up in the results. You can click the hyperlink in the description and it will actually send you to the new account name.

- If you have any video ID of that account, you can type https://www.tiktok.com/@test/video/[videoID] in your url bar to be sent directly to that video. What you put as the username doesn't matter. For exemple, https://www.tiktok.com/@test/video/7277613642206481706 will send me to mikahhlynn's account, even if I put "test" as the username. The video must be still up though.

- ID and SecUid. Each tiktok account has an ID and SecUid. These IDs do not change, even if the user changes the account name. In your url bar, typing
https://www.tiktok.com/@[ID or SecUid] will always send you to that account, regardless of its username. If the ID doesn't point to an account, that means the account associated with that ID was deleted/banned/deactivated.


To easily get the ID / SecUid of an account or a list of accounts :

- You can see the ID and SecUid of an account on your browser easily simply by installing the awemer browser extension linked earlier in the thread.

- You can use the "new awemer" python script linked earlier in the thread to create a list of ID and SecUid automatically by running the command python awemer.py -u profiles.txt -i. Just copy/paste what's in the CMD to a text file to keep that list.

- You can go on tik.fail, search for the old username of an account, open one of its archived toks, and if the tok is recent enough, if you click the tiktok logo next to the username under the video, it will send you to that account using SecUid. This is especially useful if you didn't save the ID/SecUid beforehand. I'm not sure when exactly tik.fail started using SecUid over usernames, but old archived toks won't have that feature.


One last related note, is that tiktok recycles usernames pretty fast after an account is renamed or deleted. There are more and more catfishes and scammers taking the exact same account name of old tiktok "stars" that deleted their accounts. Using account IDs or video IDs is the foolproof way to immediately tell if you're looking at the real account or not.
 

DevilDevine

Lurker
Mar 19, 2022
4
10
78
0fya082315al84db03fa9bf467e3.png
Please, Log in or Register to view quotes
This still works. It's the way the frontend now loads on the website.

If you go to a user directly from a URL (for example:
Please, Log in or Register to see links and images
): it won't give the individual URLs for all their videos on their page.
You can see by hovering over the videos it will link to just
Please, Log in or Register to see links and images
(so the bookmarklet doesn't scrape it as it doesn't match the user's page/main URL)

Now, if you go to their page through a video URL (/https://www.tiktok.com/@nickiminaj/video/7251030516940557611/) by clicking on their profile from the video, it does load all the other video URLs and the bookmarklet works as intended.

Personally, I favourite/like a video from an account I want to scrape. Go to my own tiktok profile, use the liked/favourite tab, open said video in another tab, and it's always good to go that way.

A bit convoluted, but hope that helps
 

Obicase

Bathwater Drinker
Mar 16, 2022
61
1,732
1,252
0fya082315al84db03fa9bf467e3.png
Please, Log in or Register to view quotes
Here you go, i reuploaded all the versions I had of the extension to a mega :
Please, Log in or Register to see links and images
Latest version is 2023_05_10.
I don't know if there is much use to having the older ones but it's so lightweight I figured I might as well upload them too.

Disclaimer : I'm not the dev, not responsible for anything that happens as the result of using this extension, etc.
 

clbrn

Fan
Mar 13, 2022
16
60
229
0fya082315al84db03fa9bf467e3.png
Please, Log in or Register to view quotes

Yes, it's the most convenient (database, links to videos/profiles, can download photos, can update new content easily) but It can also easily trigger temporary IP/account blacklist/ban by TikTok if not using proxy (and using it without login makes it almost useless). But the biggest deal-breaker for me - it can't download videos in HD... in your example, 4KTiktok downloads @aixaa_ys02's videos without watermark at 138MB, but her videos in HD are 451MB... personally, I prefer the quality and will download HD manually...
 

richard2480

Fan
Jun 3, 2022
20
50
97
0fya082315al84db03fa9bf467e3.png
clbrn it has downloaded some videos in 1080x1920 for me. also if you set self.check_file_exists = True on line 97 it will check if a file with the video ID already exists before downloading.

yanlichuk not sure why that is happening. it is working for me. if you can provide some more information i may be able to help.

Beakwood if you increase self.timeout_per_file = 2 on line 100 to a higher integer it should fix this. it will time out if the download takes longer than 2 * (number of files). or whatever integer you choose instead of 2
 

Obicase

Bathwater Drinker
Mar 16, 2022
61
1,732
1,252
0fya082315al84db03fa9bf467e3.png
Sorry guys, a little off-topic, but here's an update to this post I made a while ago :
Please, Log in or Register to view quotes

Tiktok seems to be trying to make it harder to find accounts by putting their ID or SecUid in the url.
Now, if you use https://www.tiktok.com/@[ID or SecUid] as your url, you'll see an empty page with seemingly no way to see the username.

However, there is an easy workaround. While the page is empty, you are technically on that user's profile page.
So you can right click on the page -> "show source code" -> ctrl + F "uniqueId" -> and it will give you the username of the account.
 

Zogbot9001

Superfan
Mar 12, 2023
23
528
737
0fya082315al84db03fa9bf467e3.png
FULL DISCLOSURE, I DID NOT MAKE THIS TOOL
I found this browser extension on 4Chan that downloads (almost all) TikToks from a user's profile, with the click of a button. I'm sharing it because it's a bit more idiot proof that the youtube-dlp methods in this existing thread.

Installation

1. Download the .zip file
2. Unzip the file
3. Do the following depending on your browser (I recommend using Firefox):


Google Chrome:
• Open the extension menu(type chrome://extensions/ in the address bar)
• Turn on developer mode
• Click "Load unpacked" and upload the root folder
• Upload the "connect-ext" folder as a second extension(a bit of a crutch to make an MV3 extension to work properly:
Please, Log in or Register to see links and images
)

Firefox:
• Open firefox extension menu(type about:addons in the address bar)
• Click on gear icon and choose "Install Add-on From File"
• Upload firefox-ext.xpi

Now, whenever you go to a Tik Tok user's profile it will look like this:

Screenshot-2023-08-15-at-03-03-37-Susu-susu_jpg-TikToke9af0946e022ed9d.png


By default, the extension will spit out all downloaded files into your Downloads folder, which you probably do not want.

Click on the extension's icon in your browser and surround the name of your desired folder in forward slashes in the download folder field.

Screenshot_20230815_031515f0506b816235851a.png

This will put all downloaded videos into a folder in your Downloads folders called SusuArchive.

Everything else from here on is fairly straightforward.

TO DO
As of now, the extension DOES NOT download all videos from a user, but it does get the vast majority of them. I do not have the background necessary to figure what is going wrong and how to fix it, so if anyone wants to contribute, please do.

DOWNLOAD
Version 2023_05_10:
Please, Log in or Register to see links and images

 

Obicase

Bathwater Drinker
Mar 16, 2022
61
1,732
1,252
0fya082315al84db03fa9bf467e3.png
Updated the mega link.
- Can download image posts
- Can download multiple videos in parallel
- Can notify of new toks of accounts in your list
- Bug fixes

Refer to readme to see how to use these new parameters.

Thanks to anon who develops this program. This is quickly surpassing every other tiktok downloader I've seen.