Thursday, December 11, 2008

PubMed & the Discovery Initiative

Why do they keep changing PubMed so much lately? It is driving me crazy!

NCBI: Discovery Initiative.

... huh?

When you hear about the National Library of Medicine (NLM), what do you think of? For most of us the PubMed database is probably at the top of the list. However, PubMed is a very far cry from all the biomedical information resources that NLM encompasses. PubMed is the default at the top of the Databases box on that Entrez page, but have you seen this Entrez cross-database search that gives a brief annotation about the other NLM databases? Try out your favorite medical subject in the cross-database search box and see how it displays the results. It's a nice way to see what might be relevant in these other databases, right?

Nobody knows about the cross-database search (not exactly a catchy name) and the entire research world knows about PubMed though. Leading the creators of the 3+ million searches per day of PubMed to explore these other databases that may be of great value to their medical research yet are completely unknown to them is the driving factor behind what is referred to as the 'Discovery Initiative' of the National Center for Biotechnology Information (NCBI), which is part of NLM.

Most of this work has been done quietly behind the scenes since the summer of 2005, preparing the backend infrastructure to handle this increased connectivity among the databases according to a Bio-IT World news article from January 30, 2006. In a Bio-IT interview published on February 1, 2006, Dr. David Lipman, NCBI Director, explains this concept further

I call it the Discovery Initiative. It’s something new. It’s been percolating. Last summer, I went out to visit Google and Amazon’s A9 [search engine] and the folks from Microsoft’s MSN came to visit. I also went up to Boston to meet with folks from the major hospitals there and MIT and Harvard. We’ve really been giving this a lot of thought. In many ways, we have had great success. Lots of people use the site and 2.25 terabytes of data are downloaded from our site everyday. And yet I find it very frustrating because we’ve connected up the scientific information in very precise and powerful ways: a protein structure to the chemical it’s bound to, to genetic data, you name it. All that is connected up. And yet very few of our users do more than very simple things with our site. It’s as if [they say], “That’s enough, I found the answer I’m looking for and I’m done.” We want them to find answers to questions they didn’t even know they had.
They're trying to make PubMed like GOOGLE?!

Not really. PubMed will never recommend you go check out WebMD & someone who has a website about a rare disease just because they have the same keywords in their metadata. Think along the lines of the suggested products when you're shopping at If you're searching for information about genetics in PubMed, in the future you probably will receive the 'But WAIT! There's MORE!' blurb about Gene, NLM's database of genes. From Dr. Lipman's February 1st interview,

The bottom line for us is discovery. We want people to make discoveries, and if we’re using up real estate on the Web page for things people don’t click on, and if we can put things on there that would have been associated information, then we should do that.
It marks a change in perspective and philosophy that will lead to constant changes in the system [in coming years].
Where was this Discovery Initiative announced to the medical library community and discussed?

The National Network of Libraries of Medicine, Middle Atlantic Region (NN/LM MAR) blog entry on February 7, 2006 cites the Bio-IT article as "the official word about the new NCBI Initiative." The October 2006 minutes of the PubMed Central National Advisory Committee note Dr. Lipman referring to the Discovery Initiative as "one of the most important projects NLM is working on." I encourage you to go to the source article for additional comments and insights about it from the committee.

There are additional brief mentions in the May-June 2007 NIH Catalyst (for NIH intramural scientists), and an NIH advertising feature in Nature from 2007, and the Nature citation was mentioned by someone in the comments to the Anna Kushnir incident on March 22, 2008. Other than these scattered bits of information, I can't find anything else. NLM Technical Bulletin has no mention of 'Discovery Initiative', MEDLIB-L has no mention of 'Discovery Initiative', and MEDLIB-L posts including either 'PubMed' or 'NCBI' around the same time as the Bio-IT & MAR blog posts don't appear to discuss it either. If you're aware of other sources that do discuss the NCBI Discovery Initiative, please post a comment and I will gladly edit my post to include them.

Why am I hearing about this from you almost 3 years later?

The changes you are now and will continue to see in PubMed are a result of the Discovery Initiative, which I heard mentioned for the first time during NLM orientation last week. I made a note of that term to research it further since I am new not only to my job but the field of medical librarianship. I thought I didn't know about something everyone else already did.

In my opinion, the existence and ramifications of NCBI's Discovery Initiative weren't communicated as clearly or as widely as they needed to be to the medical library community in 2006. It still isn't being mentioned as the reason behind the changes that are now rolling out at the front end of PubMed (remember my first shoutout about this back in May and my guide to the Chicago presentation in June?) and will continue to do so for a while. I am but one humble blogger, but believe that the more the history, background and reasons behind change are explained, discussed and understood, the better the outcome of the change is for everyone.

A tip: Start using Advanced Search now if you haven't already, the tabs are becoming obsolete for updates and are probably on their way out next year.


The old "tab" version of Limits will not be updated. Future changes will be made only to the limit feature on the Advanced Search screen.
(December 5, 2008 NLM Technical Bulletin, which I really wish wouldn't be referred to as the TB. I like Tech Bull since I'm a moderately geeky Taurus!)

If you are not already subscribed to the NLM Technical Bulletin, I definitely recommend it to know what's on the horizon.

I'll have more reflections on this, communication, and medical librarianship work culture in a post next week and encourage you to share and participate from your perspective. What do you think about this?


digicmb said...

They called it Gateway search before ...
Another usefull tool could be:

Anonymous said...

Have they actually done some evidence-based research on how and why people are using the database and their information-seeking behavior? I'm not so sure that their basic premise is correct--do medical researchers, doctors, nurses, other healthcare professionals actually WANT to go a "voyage of discovery". Mostly I see them wanting an answer they can live with in the shortest amount of time in clinical matters, healthcare administration, etc. The studies I've seen back this up. They search until they've seen enough of the literature that they feel comfortable making a decision and then they stop--even if there are still articles they should look at or information resources they should try! This is real life! I really, really like the idea of a meandering search through the literature with a skilled librarian clearing away the underbrush on a journey of discovery but that is largely a fantasy. Sigh! Too bad.

Heather said...

I actually quite liked Advanced Search for the most part, particularly the Index feature for MeSH topics that will allow you to see the number of hits for each subheading. My one frustration was that you can't hold down the CTRL key and select more than one subheading from the box. However, if you scroll down to the bottom and use the "Index of Fields and Field Values" box, and search your MeSH heading there, you CAN hold down CTRL and multi-select. How confusing is that? Maybe it's a bug?

Nikki Dettmar said...

Thanks for writing, Guus. NLM Gateway is still alive and well... in fact it is the only National Library of Medicine (NLM) portal you can search in for select professional conference abstracts. It is different from the Entrez cross-database search in that the latter is geared towards life sciences resources while NLM Gateway is geared more towards consumers... although those conference abstracts really applies to both from my perspective :)

Joykenn & dewey, I definitely can't speak to those particular questions but you raise great points and I encourage you to contact NLM directly about them!