The other day I wanted to create a PDF of all the answers and discussions of a particular Quora user I am interested in, https://www.quora.com/profile/Artem-Boytsov.
Doing this by hand would have taken me hours of work, and there’s no way I could have been sure that I really got all of them. This particular user has over 700 answers on Quora and probably well over 4000 comments discussing those answers, and many more that show up as having been deleted.
To make matters worse, Quora has a variety of ways in which it can hide answers and comments, and uncovering one kind reveals other kinds and vice versa.
To solve this problem, I wrote python script which I called
quora-expander that repeatedly looks for all of the different kinds of links that Quora offers for expanding collapsed content, and clicks on them until there are none left to click. To make sure I got all the links, I made it so that the program would try on its own to expand everything there is to expand, but after it finishes, it gives the user the chance to manually run any or all of those same commands again.
If you want to give it a try, head on over to https://github.com/swiss-chris/quora-expander where you will find the installation instructions. If you want to know a bit more about how it works under the hood, read on.
In the following paragraphs I will take a look at the various ways in which Quora collapses its content. I will discuss them in the same order the
quora-expander script executes it’s commands:
1. Lazy loading
The first thing Quora does is it only shows you the first few answers for a given user profile. To see more, you need to scroll down and more answers will keep loading as you scroll. Therefore, the first command that
quora-expander executes is
do_scrolldown(). It does this until the page stops getting longer.
2. Truncation (of answers)
The next thing Quora does is that it loads only the first few lines of an answer, offering you the possibility to show “(more)”. The command to do this automatically for all the answers on the page is called
3. Viewing More Comments
Now that all the answers are loaded and expanded, it’s time to get the comments. The first way Quora hides comments is by hiding most of the immediate replies behind a “View More Comments” button. By clicking on this button, Quora will load some more comments, but only a few at a time. The command to do this for the whole page is called
4. Collapsed Comments
Similar to “View More Comments”, Quora sometimes also shows a “View Collapsed Comments” button, which seems to do more or less the same thing as the “View More Comments” button. It loads more first-level replies to an answer. After clicking on a button of the “View Collapsed Comments” kind, you may find a new “View More Comments” button appearing after it. They both seems to do roughly the same thing. The command to do this automatically for the whole page is
5. Previews of continuing comment threads
Once a first-level comment has multiple layers of responses, Quora will hide most of that discussion under a single-line preview of the next response in the thread. Clicking on that line will load the next few lines, and so on. The command to do this automatically is called
6. Hiding More Replies
Another way Quora can hide ongoing discussions is with a “View More Replies” button. The command for clicking on all buttons of this kind is called
7. Truncation of comments
Finally, Quora will do the same thing to individual comments as it does to answers, it truncates them if they are too long and offers a “(more)” button to show the rest. The command to do this automatically is called
8. Try again
There’s one last way in which Quora might hide information, and that’s simply if you lost your internet connection before clicking on one of the buttons. In this case, Quora will thankfully show you a “Try again” button, allowing you to retry once you have restored your connection the internet and/or Quora’s servers. The command for seeking and clicking on these buttons on your page is called
So, now you know all there is to know about the various ways Quora hides content on a user profile page. To run the script yourself, head on over to https://github.com/swiss-chris/quora-expander where you will find the installation instructions.
PS, there’s one final trick I want to share with you. If you do end up creating a PDF from a Quora page, one cool feature is that clicking on the dates of any answer or comment will jump you straight to that answer or comment on Quora’s website, even if that comment is buried in a long thread of nested comments. Quora will automatically expand all the comments in that thread up to the comment you clicked on. Pretty cool!
If you liked this post, please click on the 👏 button. Thanks !