I can haz outreach? Nobody speaks for the end users.

Recently there was much buzz in the #bioinformatics twittersphere over this blog post by Sean Eddy: The next five years of computational genomics at NHGRI

It is a very nice post about some exciting prospects for the future.  The idea of planning “explicitly for sustainable exponential growth” is wise.  There will be no abatement of the flow of data at this point–it’s no longer a big bolus of one species data, or one type of project.  The taps are wide open now, and we just keep adding more taps.

I also love the idea of “democratization“.  In part, it includes:

….To enable individual investigators to make effective use of large datasets, we must create an effective infrastructure of data, hardware, and software. NHGRI has extensive experience in big data, and can lead and catalyze across the NIH….

Now, I know this is a snippet of some thoughts–there may be more to it in the actual planning meetings on this.  But it pushed my buttons because it sounds a lot like what we always hear about big data projects: build it and they will come.

It got a little better in another segment:

Spur better software development. Traditional academia and funding mechanisms do not reward the development of robust, well-documented research software; at the same time, the history of commercial software viability in a narrow, rapidly-moving research area like computational genomics is not at all encouraging….

Well-documented research software.  Sigh.  We probably read more documentation than most people. And even the good documentation can be brutal. Dated. And not particularly effective. But still–if nothing else, please reward time spent on documentation….

But what is missing for me from this–and not just this, but most of these big data types of projects–is a real commitment to outreach and support for end users.  Formal, organized, supported, rewarded, outreach.  Sometimes there is a place to write to with questions.  But we probably send in more questions to projects than most people too–and the success rate for answers varies widely.  But even when we get good answers–that’s not enough.

I know funding is hard.  We can’t fund everything.  Databases and software project have to struggle to even persist.  Curation is frequently not valued enough.  And often curators are expected to do outreach as just one of their tasks…which pushes outreach even further down the priority list.  But without dedicated outreach–formal, quality, active outreach–databases and software projects won’t have so many users, and not many effective users.   Which will make funding agencies wonder if they should keep supporting them.  Which…well, you can see where this spiral goes….

What bugs me, I guess, is essentially this: Nobody speaks for the end users. There’s really no one in these types of meeting that really speaks for the consumers of this software and this data.  I mean people who aren’t directly attached to the data production and management.   The project teams think they are thinking about the users.  They really want users.  But ur not doin’ it rite.

I would like to see outreach and end user support valued, required, and really done right.  No matter how  much hardware and documentation you throw at these projects, if people 1) don’t know it exists, and 2) have no idea how to use it, the project will not yield all the results that it could. A marker paper is nice.  But it’s not sufficient, folks. And it’s nice to have the high-end team members talk at conferences. But that reaches only a tiny subset of the users or potential users.  And another thing about that: a lot of times people are hesitant to ask what sound like naive questions to the high-end representatives of these projects.  I’m jes’ sayin.

Yes, this is fairly self-serving for me to say.  But we see the users when we do outreach.  They crave it.  They love it.  We’ve been lucky to be a part of some great projects that do outreach right.  We have seen it work.  It should be Standard Operating Procedure on software and database projects.  Not an afterthought.

13 thoughts on “I can haz outreach? Nobody speaks for the end users.

  1. Mitch Skinner

    “We’ve been lucky to be a part of some great projects that do outreach right. We have seen it work.”

    I’d love to hear more about this; it’s something I’ve been thinking a lot about lately.

  2. Mary Post author

    Oh, hey Mitch–yeah, maybe that should be the subject of a separate blog post–how to do it right, what people want….

    We have a bunch of thoughts on this too.

  3. John Doe

    When graduate students are writing the software and are being pressured to publish or graduate, how can you expect well-supported software? I am sure the thesis committee will not consider how well-supported the software is. IMHO user’s need to have more tolerance for the grad students behind these projects.

  4. Mary Post author

    John–that’s exactly the problem. There’s no mechanism for that support to exist in the current system. Maybe it needs some exploration to come up with some creative structure. Maybe the project needs to include a support person time allotment, or coordination and dollars for someone who wants to be a science teacher, or technical writer, or something….

    But ok, unsupported software gets done, put out there, not used. Is that a good value for the grant dollars?

  5. Melissa Cline

    Mary, I for one would love to hear your thoughts on how to do outreach right!

  6. Mary Post author

    @Melissa: for a long time we’ve had relationships with various major databases and software providers to do seminars, represent them in our booths at conference, develop materials that we can use and that we know other people use in workshop settings. We even tweet stuff for them–last week I got a bunch of people to go to the UCSC Neandertal portal with a tweet! I got the announcement for that from the UCSC announcement list, which is great–UCSC has awesome mailing lists. But the only people on the announcement list are the people who already know there are announcements and announcement mailing lists.

    People seem to really appreciate getting access to those materials and spreading the information out there.

    I do think we should have a longer post on what we have specifically done that has worked well.

  7. Trey

    NHGRI has made some good efforts in the past at funding outreach and training. I would like to see more specific focus on this in these long range plans though.
    UCSC is a prime example of doing outreach well with excellent mailing lists and very good support, in addition to the workshops, conference outreach and tutorials they do through us and their own. Some aspects of outreach and training work better than others and in our experience we’ve learned a few things that work better than others and many that work well.
    I think one of the big issues for most resources though is funding. As a commenter mentioned above, the funding institutions and developers in the larger discussion are going to need to give a high priority for outreach and training.
    There are some good discussions going on about funding that I’ve mentioned last year:
    (De)funding Databases
    (Re)funding Databases

  8. Mary Post author

    The Blind Scientist made this point recently too–I did a post on that:

    http://blog.openhelix.eu/?p=4062

    “Bioinformatics needs investment

    And a lot of it, but not to buy larger machines, or to create big departments or institute. Not even research investment. Bioinformatics needs investment in people, in courses, in workshops, in good practices for software development, in a knowledge base.”

    But even NCBI cut way back on outreach some time ago. I know it still exists on some level, but their very popular Field Guide trainings were drastically reduced. As the software at NCBI expanded and the interfaces all changed.

    Support should have sustainable exponential growth as well. There are always new people who need training, new project directions that require new explorations, and new data types come out all the time. Papers are not the right mechanism for this world today.

  9. John Doe

    I apologize for the pessimism. Look at a few recent tools papers in NGS (Hydra-SV and Cufflinks, both in Genome Research I think), for which the papers were interesting not just because of the method. For many, the funding dollars are for answering the biological question, not the tool. Graduate students are not good software developers, but when is a research lab really going to shell out for $100K+ for a good database developer or software engineer (market costs)? UCSC and Galaxy are good examples of tools with support and outreach.

    What about a solution where those who use the software give some of their grant money to the tool developer group? A few users each giving a small amount would add up!

  10. Mary Post author

    Not to worry, John, I totally agree. The students on the projects should get support for this–via the lab director somehow, or some other mechanism. I’m open to ideas there.

    Although I would like developers to write better documentation, sometimes they aren’t the right ones to do it all. Some of the code documentation, sure, they have to do that. But they don’t think like naive users, end users, as much. They are too close to the heavy lifting to really consider what new users face, actually. And doing training is different, and developing the right materials that users need is another set of skills, really.

    A lot of people have tried ways to raise money for their database/software projects. Subscriptions and licenses generally don’t work well, although there are some successes–that’s rare. Another thing that is often tried is getting the user community to do stuff. Also not generally rewarded for the time it takes, and peters out pretty quickly. Updates are a big problem there too.

  11. Isabelle

    Great post.

    Ever tried conducting usability tests? I was surprised at how often users do not say what they think (‘what I really mean is ‘ ) and do not do what they say (‘I clicked on that button ‘). The experience was invaluable because it gave immediate, tangible feedback. I would love to see funding allocated to usability tests.

  12. Mary Post author

    @Isabelle: no, we haven’t done formal usability tests. That’s not in our remit. But we do get a lot of feedback that goes back to developers as part of what we do. For example, when we were going around first doing trainings for the UCSC Genome Browser, one of the things we were hearing was that the graphics were too small for users. We brought that back to UCSC and they added a feature to change the sizes of the items in the browser.

    Similarly, after the early release of in silico PCR, people asked for Tm values. So they added that as well.

    But I agree–usability would be another excellent piece of this.

  13. Pingback: Friday SNPpets | The OpenHelix Blog

Comments are closed.