How to expand all answers and comments on Quora with Python and Selenium

The other day I wanted to create a PDF of all the answers and discussions of a particular Quora user I am interested in, https://www.quora.com/profile/Artem-Boytsov.

Doing this by hand would have taken me hours of work, and there’s no way I could have been sure that I really got all of them. This particular user has over 700 answers on Quora and probably well over 4000 comments discussing those answers, and many more that show up as having been deleted.

To make matters worse, Quora has a variety of ways in which it can hide answers and comments, and uncovering one kind reveals other kinds and vice versa.

To solve this problem, I wrote python script which I called quora-expander that repeatedly looks for all of the different kinds of links that Quora offers for expanding collapsed content, and clicks on them until there are none left to click. To make sure I got all the links, I made it so that the program would try on its own to expand everything there is to expand, but after it finishes, it gives the user the chance to manually run any or all of those same commands again.

If you want to give it a try, head on over to https://github.com/swiss-chris/quora-expander where you will find the installation instructions. If you want to know a bit more about how it works under the hood, read on.

In the following paragraphs I will take a look at the various ways in which Quora collapses its content. I will discuss them in the same order the quora-expander script executes it’s commands:

1. Lazy loading

The first thing Quora does is it only shows you the first few answers for a given user profile. To see more, you need to scroll down and more answers will keep loading as you scroll. Therefore, the first command that quora-expander executes is do_scrolldown(). It does this until the page stops getting longer.

The three dots at the bottom of the last answer indicate that more answers are being loaded

2. Truncation (of answers)

The next thing Quora does is that it loads only the first few lines of an answer, offering you the possibility to show “(more)”. The command to do this automatically for all the answers on the page is called do_show_more_of_answers().

Clicking on “(more)” will show the rest of this answer

3. Viewing More Comments

Now that all the answers are loaded and expanded, it’s time to get the comments. The first way Quora hides comments is by hiding most of the immediate replies behind a “View More Comments” button. By clicking on this button, Quora will load some more comments, but only a few at a time. The command to do this for the whole page is called do_view_more_comments().

The “View More Comments” button will load a few more top-level comments

4. Collapsed Comments

Similar to “View More Comments”, Quora sometimes also shows a “View Collapsed Comments” button, which seems to do more or less the same thing as the “View More Comments” button. It loads more first-level replies to an answer. After clicking on a button of the “View Collapsed Comments” kind, you may find a new “View More Comments” button appearing after it. They both seems to do roughly the same thing. The command to do this automatically for the whole page is do_view_collapsed_comments().

The “View Collapsed Comments” button does roughly the same thing as “View More Comments”

5. Previews of continuing comment threads

Once a first-level comment has multiple layers of responses, Quora will hide most of that discussion under a single-line preview of the next response in the thread. Clicking on that line will load the next few lines, and so on. The command to do this automatically is called do_expand_hidden_comments().

Clicking the darker gray area at the bottom of this thread will fully load that last comment, as well as more comments that may follow it

6. Hiding More Replies

Another way Quora can hide ongoing discussions is with a “View More Replies” button. The command for clicking on all buttons of this kind is called do_view_more_replies().

Clicking on “View More Replies” will load more comments in this thread

7. Truncation of comments

Finally, Quora will do the same thing to individual comments as it does to answers, it truncates them if they are too long and offers a “(more)” button to show the rest. The command to do this automatically is called do_show_more_of_comments().

Clicking on “(more)” will show the rest of this comment

8. Try again

There’s one last way in which Quora might hide information, and that’s simply if you lost your internet connection before clicking on one of the buttons. In this case, Quora will thankfully show you a “Try again” button, allowing you to retry once you have restored your connection the internet and/or Quora’s servers. The command for seeking and clicking on these buttons on your page is called do_try_again().

Clickin on “Try Again” will attempt to load the same content after the last failed attempt

So, now you know all there is to know about the various ways Quora hides content on a user profile page. To run the script yourself, head on over to https://github.com/swiss-chris/quora-expander where you will find the installation instructions.

PS, there’s one final trick I want to share with you. If you do end up creating a PDF from a Quora page, one cool feature is that clicking on the dates of any answer or comment will jump you straight to that answer or comment on Quora’s website, even if that comment is buried in a long thread of nested comments. Quora will automatically expand all the comments in that thread up to the comment you clicked on. Pretty cool!

If you liked this post, please click on the 👏 button. Thanks !

I write about spirituality and personal development.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store