Where’s Waldo? Can You Find Your Reference Material?

Feature Story – By Alison Quinn and Shawn Thomas

Finding reference materials can be a bit challenging.

We’ve all heard of the series of children’s puzzle books called Where’s Waldo, where you must find one item in a crowded and confusing landscape of similar images. Finding reference materials for technology and the field medical teams on a SharePoint site was becoming very much like one of those puzzles.

Phase I: Technology Reference Material

Our challenge started with technology reference material. Updating reference material associated with the global use of technology has inherent challenges with respect to version control, timeliness, translation and distribution. We started with printed materials, then transitioned to digital versions distributed electronically. These digital versions were also simultaneously posted to a SharePoint site, eliminating the need for printing and shipping. Unfortunately, version control, organization, translation and manual distribution remained as key challenges. Another hurdle was ensuring that the help desk could access approved content for home office and field users seeking support.

New Platform

We initiated a search for a platform to manage reference material, including functionality for version control and distribution. Inkling was identified as an application that could instantly disseminate material electronically to defined distribution lists. Content could be aligned to specific roles, therapeutic areas and markets, so end-users would only see content applicable to their roles. Additionally, because the content resides only in Inkling and cannot be exported or printed, version control could be maintained, with new or updated content pushed to the user upon sign-in, overwriting any existing previous versions. Another benefit is that the pushed content, because it resides locally, enables user access even when offline.

After initial training by the Inkling team, we created templates and custom functionality widgets for use in development work streams and conversion of existing content. The content was designed using a logical, consistent format. An eBook starts with an introduction to a topic or concept, followed by step-by-step instructions for the process, with asides highlighting key pieces of information interspersed throughout. The material is laid out in an easily digestible format that aligns to Bristol-Myers Squibb standards. This structure gives users familiarity, so they can anticipate the experience they will encounter each time they launch the application.

This approach allowed for a multi-modal learner experience for hat had traditionally been one-dimensional reference material.

This approach allowed for a multimodal learner experience for what had traditionally been one-dimensional reference material. All Bristol-Myers Squibb technical reference material was converted to this digital, eBook format and made available to the worldwide teams. This new format allowed for the design of materials that could be easily accessed and searchable, as well interactive, while ensuring content aligned to a consistent user experience for all stakeholders globally.

The customized platform enabled the design of tools that align to the needs of the content, providing a variety of options to engage users. The following includes just a few of the tools that have been utilized so far:

Checklist functionality so users can track their progress through content.
Videos explaining how to perform a task.
Search functionality helps users quickly identify the specific guide and page within a guide to find what they need.
Show/hide widget gives users the opportunity to display or reveal content that may interest them.
Cross-linking within Inkling and to non-Inkling resources can highlight material the user may not necessarily be aware of, driving further usage and connectivity.
Knowledge checks and flashcards allow users to practice their understanding of material
Simulations allow users to rehearse a process without having to step into their live environments.

Users can view their assigned material on multiple devices: computer, phone, or tablet. The application is responsive to each specific platform, reformatting the material to fit on the screen. Users can also download most content assigned to them for offline use, giving them access to their material at any time or place.

Now that there was a mechanism to deliver and update the digitized reference material, we had to address the challenge related to translation of content for a global audience. Exporting content into Word for translation, then reimporting into the library after translation was complete helped to make this easier to manage.

Metrics

We use metrics to evaluate our new platform and content in a variety of ways, both formally and informally. Using the analytics built into the Inkling application, platform and document usage are evaluated on a quarterly basis. These reports track data such as most-visited content, time spent on individual pages and sections, the breakdown of page visits by therapeutic area, and the frequency of help desk visits to support and answer field-user questions. This valuable data helps us identify overall trends in usage, as well as potential causes of upswings or downswings.

As mentioned above, the Bristol-Myers Squibb help desk also uses content developed in Inkling to support calls from the field and resolve help desk tickets. The magnitude of usage by the help desk is tracked and reported as part of the formal quarterly reporting, providing insights into where users are having issues with technology.

Knowledge check and assessment tools are available to gauge end-user retention of content. Results from these tools can be tracked by the administrator. For the technology reference content, assessments have rarely been used, with more focus placed on analyzing page views and duration of visits.

Cost Savings

With a complete conversion of our technology reference materials from printed materials to digital format, we saw an immediate cost savings by eliminating printing and shipping costs. Cost savings continue to grow as updates to existing material are distributed through the application. In addition, if material needs to be distributed to a new audience, we simply assign the content to those new users to grant immediate access. Content is assigned by role and/or department based on our employee roster. New hires will automatically receive the content relevant to their role upon hire. Material cost savings for shipping and printing are estimated to be at least $200,000 per year.

Users automatically receive an alert when an update to their material is available through the platform, reducing the need and cost to develop and distribute a communication. That amounts to a savings of over $2,000 per communication per update.

There is also a cost-savings realized from an efficiency standpoint. Because the reference material is fully searchable with a Google-type interface, users spend less time navigating to and looking for content. The version control feature of the application assures all material accessed is current and up-to-date. Content within Inkling links out directly to other, non-Inkling resources, eliminating the need for users to navigate to another application to find related content.

Because the reference material is fully searchable with a Google-type interface, users spend less time navigating to and looking for content.

Phase 2: Field Medical Therapeutic Training

Field Medical (FM) teams currently access training materials on SharePoint, navigating to multiple sites and folders to complete training. The Oncology FM teams needed training that was easy to access, engaging, available offline and could provide analytics to improve upon future content. The platform had been used successfully for the distribution of technology reference materials, so we evaluated it as a training repository. The application had been available to all of FM for several years, but had been underutilized, so there was no additional cost to leverage the platform.

Inkling provided a way to organize all content in one place, becoming our training solution for Oncology FM teams. Content is currently being created for the Oncology FM teams to support onboarding new hires and in preparation for new indications. The content is layered with multimedia and built-in interactivity to increase engagement and retention of concepts. Pages of content can be linked, allowing for information sharing across teams, as well as version control. The Oncology FM teams are able to access content on-demand, with almost 400 page views in the past three months. Learners are revisiting content pages an average of once a month.

Future State

New materials are being created to take advantage of advanced features in the platform. Assessments and flashcards are being built nto new curricula. These reinforcement techniques allow us to utilize the additional functionality of Inkling to identify any training gaps that could be addressed by improving or adding additional content. The application analytics also allow us to assess resource utilization and potentially remove content that is not being viewed. Future trainings will look to leverage more interactivity. The training teams continue to work closely with Inkling to understand new functionality and incorporate it quickly to keep training engaging and effective.

Alison Quinn is oncology field medical portfolio trainer for Bristol-Myers Squibb. Email Alison at Alison.Quinn@bms.com. Shawn Thomas is director of technology training solutions for Bristol-Myers Squibb. Email Shawn at Shawn.Thomas@bms.com.