Uncategorized

stack overflow reading comments

What a very bad day at work taught me about building Stack Overflow’s community

Hi, my name is Sara Chipps, first time Stack blogger, long time Stacker (I’ve always wanted to say that!). I’m the new Director of Public Q&A at Stack Overflow. I’ve been at Stack for a year now, and I’d like to share with you one of my worst days at work, and what it taught me about the Stack Overflow community.

A little about me: I’m Stack Overflow user #4140. I was in the beta and one of the first people to ask a question on the platform. Stack Overflow has been a big part of what I do for a long time. I’ve been an active member of Q&A, a participant on Area 51, and a lurker on Worldbuilding, Cooking, and Code Golf. I’ve looked for new opportunities on our Jobs board and placed job ads when hiring great talent at companies I helped run, like Jewelbots or GDI.

The thing that really connected me to Stack Overflow and its community, however, is the simple fact that I’ve been a software developer for 18 years. I cut my teeth on MS SQL and Data Warehousing. I moved on to C# and .NET in 2006. I was a .NET MVP for 2009 and 2010 before switching to JavaScript, Node.js, and building Nodebots in 2011. I love JavaScript with all my heart, warts and all. Since joining Stack Overflow’s engineering management team in 2018, I’m back in the world of .NET and on the board of the .NET foundation helping the framework build the future of the internet.

I care a lot about representation in technology, and a future where people from underrepresented groups in technology are hired and succeed at the same rates as their peers. Being a part of the team helping to guide the direction and growth of community is an incredible honor.

We’ve been working on exciting things the past few months to make the site more welcome, diverse, and inclusive. To kick off these changes, we started with the tools that our moderators and power users rely on to make Stack Overflow the best site for developers online. The thing I’ve seen our Community Managers push for the most is updating these dated tools, some of which haven’t been touched since we first launched! The team formerly known as DAG (Developer Advocacy and Growth), now part of the Community team, started by rolling out the Tag Synonyms Refresh and the improved Moderator Dashboard. Paying down this debt will continue to be a priority as we work to get our mods best-in-class tools to manage their communities.

The second thing we are prioritizing requires a bit of a story. When I joined Stack Overflow almost a year ago, I was blown away by how kind and generous all my coworkers were, and the engineering team is no exception. As engineering manager for the team charged with working on our Talent product, I got to work closely with brilliant people I really respected. It was a treat to be collaborating with lifetime learners and natural teachers, the kind of engineers that you would want on any team.

About three months in, on a Friday afternoon, we introduced a new company-wide policy that I felt was relatively benign. What happened next was that, from my point of view, the engineering team completely lost it. No one agreed with this policy, and they made it known over seemingly hundreds of Slack pings. After an afternoon of going back and forth, I walked away feeling emotionally drained. What had happened to my amazing coworkers that were so kind and wonderful? I felt attacked and diminished. It seemed people weren’t valuing my work or my judgment.

I went home for the weekend and stewed in my frustration. I replayed everything that happened in my head and each time got more frustrated with the way people reacted. When Sunday rolled around, I decided I wanted to look back at our Slack conversations and see which one of my coworkers was being the rudest and the most unreasonable. I wanted to give them direct feedback that they had hurt my feelings.

As I went back through that Friday afternoon chat log, I was shocked to see that no one had been hurling insults. There was no one saying mean things about me or attacking my efficacy directly. In fact, what I found was that people had some well put together arguments about why they felt this policy was a bad idea. The entire engineering department definitely made their criticisms known, but I didn’t find people questioning my ability as a manager, throwing around insults, or saying anything that that illustrated why I was feeling so targeted.

That was when something became crystal clear: my coworkers hadn’t become monsters, they were still the kind and caring people I thought they were. The monster in this case is not one person, it was created when lots of people, even with great intentions, publicly disagreed with you at the same time. Even kind feedback can come off as caustic and mean when there is a mob of people behind it. No matter how nicely they say it, when a large group of people you really respect publicly challenge something you’ve done it can feel like a personal attack.

When I realized this, some of the confusion I had seen about unwelcomeness on Stack Overflow started to make sense. In our developer survey results we read things like this:

  • Caustic community for new users. There is no excuse for not being kind!”  – 6 years coding
  • It feels too scary and unaccessible for new developers” – 3 years coding
  • People could be less brutal” – 6 years coding
  • The attitude is not beginner friendly. Askers are expected to have done a lot of research before asking a question (re: both question format and content), even if they are completely new to the community or topic. Not everyone can understand or even know to look for documentation when they’re completely new to programming.” – 12 years coding experience

However, when our more experienced users hear this feedback they ask us to provide them with definitive examples of WHERE EXACTLY people are being unfriendly? There isn’t a lot of name calling or anger, why are they being accused of being unfriendly?

People tell us they are afraid to participate because of how mean their peers can be. The way the system is currently built, when you ask a question that could use some editing or is a duplicate, a bunch of people come out of the woodwork to tell you you’ve done something wrong.

They could say it in the most neutral possible way, but no matter how you approach it, a dozen people pointing out your errors feels terrible. Not only does it feel terrible, but it can also be not beneficial for overall content quality, not to mention an ineffective way to get someone to improve their question. There is also a big yellow box that gives you the names of engineers that voted your question closed or deleted. That in itself can feel really bad especially as our high rep users skew toward more experienced and respected engineers.

On one hand, our more experienced power users tell us they feel called out for being unfriendly even when they are just trying to be helpful; on the other, our newer users and people that don’t participate tell us they think Stack Overflow is scary and they are afraid of judgement. In the past, we’ve prioritized getting rid of unfriendly comments. We’ve seen improvements there, but we still hear people feel targeted even when there aren’t unfriendly comments. This problem is on us and it’s because of how we designed the question asking and closing process. People are using the product as it was designed and as a result people feel called out or, even worse, discouraged from ever asking a question again.

Over the next few quarters, we’re going to be taking a step back and re-evaluating how we deliver feedback to users about their questions. We want to make sure people are getting necessary feedback without feeling called out or publicly embarrassed. We will be working on new paths to improve content quality and reduce friction between people. Our goal is to have the question asking process be painless and beneficial for new users and Stack Overflow veterans alike.

By improving the way people give each other feedback, we can improve question quality without putting the burden on our users to police the website. We will empower our long time users to become mentors and teachers in order to bring the spirit of Stack Overflow back to what it was in the beginning, a place where people come to share and learn. By thinking hard about how we give feedback, we’ll help people learn instead of driving them away. We’ll get more people involved and improve question quality.

Myself and the community team are really excited to improve the experience that all levels of coders have on Stack Overflow, from new users that are learning front-end for the first time to our respected moderators who have been coding for 20+ years. We all have ideas on how to make the system better. The great news is we have experienced researchers, data scientists, and an amazing product manager that will be gathering feedback from us, the community, and many other places and partners to make educated decisions about solutions.

We think the world of our community, and are excited to hear what you think of improvements as we make them. We’ll make sure, as always, to keep you posted.

Related Articles

Comments

  1. Hi Sara,

    Your blog post gives me some hope for StackOverflow future. You have identified a central issue with the new user experience…

    It is horrible.

    I am a new user of StackOverflow, but not new to coding, as I have been coding since 1979. Finding answers to questions on StackOverflow via Google is great, but asking questions is a terrible experience, and answering questions is a terrible experience. Too many members behave as though the same level of EQ as Sheldon of ‘The Big Bang’ fame and do not seem able to interact in a conversational manner.

    I answer a lot of questions in FaceBook groups where we seem able to give answers without presenting the feeling that we are ‘up ourselves’. No one likes to feel an idiot for asking questions, I certainly don’t.

    It may be that moderators need to interact with the answers to or comments on questions from other users before they are displayed. Essentially, I feel that a customer service ethos should be the norm and that moderators need to display both technical skills and empathy in interactions with users.

    1. Farzad says:

      I am so much in agreement with you ! the website approach towards the newer and newer questions get more and more horrible, more trolly ,snarky, obsessive and oppresive

  2. Great post. it is really clear how the situation is perceived (and I didn’t think about it in previous situations, for be honest).

    What I would like to share is , for example, about the comment setion in each Q&A. If the question is duplicated, the workding should be different, like: “this is a very common problem. You have check here the differents approaches” or some on these lines.

    If the question is closed/deleted because it was a duplicated, the page should say something like “This question was already asked and the most accepted solutions can be found here” > list all questions that has accepted answers. “Since your question already exists, we only have those questions that aren’t already asked before” – i.e, a cleaner message is better. IMHO.

    I think that by saying “This question was removed for moderation purposes” or some in those lines, a new user would feel worse instead of giving him/her real feedback about why your question was removed.

    Side note, I think you omitted a word at the end. Quote:

    “We think the world of our community, and [myself] are excited to hear […]”.

  3. Emily Cain says:

    I don’t know if this is under consideration but it might be helpful to have a “sandbox” forum where newbies post their questions, similar to how code golf meta has a draft/staging area for people to get feedback on their challenges. New posters’ questions would default to going there and either be pointed to resources to answer their question, or if it has the potential to go on the main StackOverflow site, get feedback on how to improve it.

    I think there’s a fundamental conflict between how StackOverflow moderation is geared towards each question being unique and useful to the world at large, versus being useful to newbies who probably don’t have the work background or context to understand where their question fits into the overall corpus of information on the languages and frameworks they’re using. That’s a lot to ask of anyone at any level, really, and people who just need help with a tutorial or their first web app are not served well by it at all.

  4. silver says:

    The emotional investment required to stand there and take critical feedback is quite large. It’s difficult, especially when you are not sure if your answer is correct or helpful, but you are taking a risk, putting yourself out there, and trying to be helpful when no one else has yet. Knowing that stackoverflow is working on keeping that feedback onslaught from being a barrier to entry is nice to hear. Keep it up. 🙂

  5. My personal favorite is of the form “This question has been asked and answered before. [no link provided]”.

    What if instead of rating questions, people either answered the question directly or with a link, or didn’t reply at all. That way, all the replies one saw would be helpful.

    1. Emily Cain says:

      IMO marking a question as “duplicate” should require the person reporting it to post a link to the supposedly duplicated question.

    2. Ken Walker says:

      My “favorite” is similar “This question is closed as duplicate. It has been asked and answered before. [but the link provided DOES NOT ANSWER MY QUESTION]”. What makes me especially crazy is when the moderator(s) that vote to close my question don’t display any skill in the language I am asking (I check moderator profiles). I add comments to explain how it’s not a duplicate, and that their link doesn’t answer my question…usually to no avail. I don’t take it personally. But, in the end, I still don’t have an answer to my question. Frustrating.

  6. Andrew Morton says:

    If new Stack Overflow users were introduced to some of the conventions on SO, perhaps there would be less culture-shock.

    I asked if something could be done about that with [Where is the conventions help page?](https://meta.stackoverflow.com/questions/368159/where-is-the-conventions-help-page/368176#368176) but there was very little response to it.

    I think perhaps your “The Newbie Experience” illustration shows that it is important that the user does stick around immediately after asking their question, otherwise even an almost well-written question can be closed in minutes if there is no clarification from comments asking for just one extra piece of essential information. But no-one has told them that.

  7. Travis J says:

    Very insightful post. This frames a lot of issues in an elegant manner.

    I have been a long proponent of the exact outlook you seem to provide here, and am eagerly awaiting the next steps.

    The closure process itself has been described as attempting to be welcoming when it was first designed, and adjusting that process from its current state to match the current environment is effort well spent.

    “We will be working on new paths to improve content quality and reduce friction between people.”

    Yes. This please.

    I think reducing friction between user and content removal can be achieved by

    – setting realistic expectations about the platform for users
    – using the interface to explain what is happening while it happens
    – expediting the process of content removal
    – removing any vague descriptions of what constitutes topicality

    Progress in any of these areas will be beneficial. After all, the telephone only stared with single digit numbers, and look at where we are now.

  8. Joshua Hudson says:

    So what was the policy?

  9. Gregory Roby says:

    Hello, I think this is a great post but see one item missing from the Newbie Experience diagram [maybe it is part of the aftermath]. Often there is also a comment that 100% answers the original question/problem that the original poster had, as well as answering anyone else that finds the Stack Overflow post on Google and wades through all the negativity to look for the answer (usually a page or two down). Have seen that at least a half dozen times for various odd issues over the years. Generally happens on questions closed by the moderator.

  10. Thanks for sharings that, Sara. =]

  11. As you pointed out in your own experience, the problem is not the community in general, as most gave direct feedback. The problem is the individual reading the feedback not reading the actual words that are written but projecting their own emotional response in place of the words that are written. You can’t prevent that, nor can you help that along – it is a problem the individual must conquer themselves. The engineers in your case were right to question the accusation of them being unfriendly and to request exactly where they were doing so because it wasn’t the case.

    Blaming the community in general for being unfriendly when it’s not the general case will cause community blowback at the accusation because the blame is generally on the individual for replacing the words that are written with their own emotions, which is exactly subjective, uncontrollable, and not very conducive to a rational conversation.

    With that in mind, I think it’s great you’re looking at improving the experience, and would focus on helping newcomers get familiar with the expectations of the community rather than change the community to the expectations of new users, which would vary significantly – to the point of being an impractical goal to meet.

  12. Thanks for facing your challenges, sharing them, and using the pain to improve SO.

    The new(?) guided mode looks like it’s part of this effort. Would it be possible to have guided-mode-level feedback in the FAQ box on the right, or to show an example form box being filled out as user steps through the guided form.

    Or checkboxes of common criticisms or question-quality sufficiency criteria – which start out un-checked, and can be half marked by a user, with the implication that mods / users / the community will address the question on these aspects.

    e.g. on-topic (answerable, not recommendation, not meta, not opinion?), etc.

    It feels like you could train users to follow the checklist, but it requires bouncing around the page. Not some visually linear progression. Maybe there needs to be a top-down (category/tags, topic, question) vs bottom up flow (question–draft, add topic, add tags)

  13. TylerH says:

    Welcome to the role, Sara. I hope you have great success in the role and in the tasks you’ve set out for yourself here.

    Regarding “This problem is on us and it’s because of how we designed the question asking and closing process”, I have to disagree, however. While that’s a big pain point, the question asking/closing process design is not the cause of the problem.

    The cause of the problem is the perception of Stack Overflow. It’s viewed as a “get help for my problem” resource, when it was originally billed as a “helpful repository of useful solutions to programming problems”. Part of that is inherent in the Q&A format vs a Wiki, article, or blog format, and I’m not sure if that part can ever be solved. But part of it is due to the fact that the site allows, and in fact strongly encourages, strangers or new users to post their problem and expect a fix for it, regardless of any other criteria or quality standards.

    If you want to talk about paying down debt, I think an important thing only someone in your role can do is address a long-welling issue head-on: make a public announcement on the company’s position regarding how they view Stack Overflow public Q&A. Is it for professionals and enthusiasts? Is it for beginners? Both? Whichever that is, what does that mean? Does Meta have a say in shaping the kind of content they want to accept on the site (like every other SE community)? Or is SO a special place where Meta doesn’t really matter, just like Whose Line with Drew Carey?

  14. This article confirms researches like “The Role of Positivity and Connectivity in the Performance of Business Teams” which states that on average, humans need about 6 compliments or positive feedback to be able to handle a negative feedback/criticism.

    I do agree that the amount of feedback can be overwhelming on Stack Overflow and I love the way you illustrated that!

  15. Ismail says:

    I think downvote should not be allowed without a comment explaining why it was downvoted. This way the questions and answers will improve drastically.

  16. Hilary says:

    I agree with Shawn’s comment. I find the recent trend in society to be overly concerned with others’ emotional state troubling. There is an increasing tendency to preamble sentences (“In my opinion”, “I feel that”, etc.) and excessively caveat what you say in order to make it more clear that anyone listening should not be hurt or offended.

    This trend runs counter to clear speech and conveying meaning. I would refer anyone to George Orwell’s excellent 1946 essay “Politics and the English Language”, and ask them to consider how an update to this essay would certainly include an additional list element beyond Orwell’s:

    1. Never use a metaphor, simile, or other figure of speech which you are used toseeing in print.
    2. Never use a long word where a short one will do.
    3. If it is possible to cut a word out, always cut it out.
    4. Never use the passive where you can use the active.
    5. Never use a foreign phrase, a scientific word, or a jargon word if you can think ofan everyday English equivalent.
    6. Break any of these rules sooner than say anything outright barbarous.
    7. Trust your audience to manage their own emotional reactions

    Politics and the English Language: https://www.npr.org/blogs/ombudsman/Politics_and_the_English_Language-1.pdf

  17. Joseph Brown says:

    This is a brilliant and novel insight into UX design and the human mind. However, a minor quibble: “Myself and the community team” is terrible grammar!

  18. Shadow Wizard says:

    Shining light during a dark era. Huge kudos to you for being so honest, and coming with true heart to help both sides.

    If I may give advise: bring back the Mentorship project. It was awesome, and just what we need. It let power users give feedback, even negative, to new users in a personal and non-intimidating manner. I took part and enjoyed every moment. Having two questions asked in a better way because of the feedback I gave felt better than getting 1000 upvotes.

    Cheers, and good luck!

  19. One of the sources of frustration I see with new users is that their questions often get closed rapidly by high-rep users. However, I’ve seen a lot of questions that have several votes to close, even when I think it’s a good (or at least not bad) question. I really wish there was an option to vote AGAINST closing a question, so there has to be a consensus among the high-rep users who view it.

Categories: Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s