A Twitter profile archival tool.
This update is all about threads⚡! Three main large changes:
--depth
and -d
for archiving threads.--multi-threading
and -t
to specify number of threads to use.parent_id
to tweets.json
which tracks direct tweet responses.threading.py
which manages thread workers and queues.pages.py
which abstracts away a page on Twitter. Useful for multi-threading.
TwitterPage
used for representing a page in Twitter.TwitterBio
which is used for fetching and obtaining bio pages.TwitterThread
which represents a tweet thread page.TweetExtractor
which attempts to fetch all available Tweets on a page.Scroller
which abstracts away scrolling metrics.hit_more_replies()
and get_recommend_tweets_height()
.Many thanks to those who have contributed! This release attempts to modularize and add a few changes/guidelines to help aid in the community effort in this project. It also renames our project to make finding from existing tools easier.
src/
.bin/
.BirdWatch
-> Twitter Bird Watcher
(TBWatcher
)logging
module.--login
for giving users login--url
support for single profile scraping