June 19, 2017

WWDC 2017 – Apple Takes AR, VR and Machine Learning to New Heights

Image of Apple monitor phone keyboard mouse and laptop


Apple isn’t always first to the party, but it generally makes a grand entrance when it arrives; this year’s annual Worldwide Developer Conference (WWDC) was no exception. Thousands gathered recently in San Jose, California, just outside Apple’s headquarters in Cupertino, to hear about its latest software and hardware advances. Among other things, Apple announced substantial software updates to its Watch and iPad, and also unveiled ARKit — an augmented reality developer toolkit; Business Chat — a chat tool that uses iMessage; and a Siri-powered speaker — think Amazon’s Echo — called HomePod.

In this POV, we summarize Apple’s latest software and hardware announcements and identify ways pharma marketers may leverage Apple’s new tech to reach wider audiences and provide better services and experiences for those audiences.


WatchOS 4: Hyper-Personalized User Experiences

Apple Watch’s latest operating system, watchOS 4, features a number of health-related improvements, some coming paired with the iPhone’s iOS 11:

  • GymKit— a new fitness app that will allow Watch wearers to sync their Watch with cardio machines to transmit workout data — including machine settings like incline and speed — directly to the Watch, simply by tapping through near field communication (NFC). In the future, based on Apple’s patent activity, we may see similar tracking built into exercise machines to automatically measure weight and reps.
  • Communication with low-energy Bluetooth devices — using Core Bluetooth, the Watch can connect with health and activity trackers (such as the Zepp tennis tracker) and medical devices (such as the Dexcom continuous glucose monitor), without relying on a phone being present.
  • Predictive recommendations — Siri intelligence is now smarter and preserves profile information like location, and personal and activity-based data, and synchronizes it across all Apple devices to provide a higher level of personalization. For the Watch, there’s a new Siri-powered watch face that uses machine learning (which Apple calls Core ML) to provide predictive recommendations on managing your day, such as driving time to your next appointment; gamification elements like interactive, animated achievement notifications; and fitness challenges that are automatically adjusted based on prior activity. These Siri-powered features could be used to help patients adhere to medication regimens or even – through gamification – get e-coupons for over-the-counter medications like allergy pills or pain relievers they regularly purchase.

With the proper privacy/security permissions in place, we could use the Watch’s locational awareness + simultaneous localization and mapping features to detect when a patient arrives at a hospital and provide indoor wayfinding. We could use NFC-based smart packaging for prescribed medication to help drive adherence with a simple “tap-to-refill” feature similar to Amazon’s tap-to-order Dash button.

Image of apple health products

By blending users’ activity and device data with data gathered from a variety of other sources, we’re in a better position than ever before to maximize patient-centricity and use cognitive computing — in which computers employ data mining, pattern recognition and natural language processing — to enable predictive management and sharing of health information. One example of an app that’s already doing this Migraine Buddy, which gathers behavioral information directly from the user as well as sources like weather forecasts to alert migraine sufferers so they can take preventive action before a headache sets in.

Enhanced iPad Capabilities

One reviewer of the latest iPad Pro said it represents “a maturation into a truly versatile and powerful computer, thanks to Apple’s usual mix of tightly integrated hardware and software upgrades.” iPad’s updated capabilities include:

  • Increased screen refresh rate of up to 120 times per second, maintaining fluid motion when performing complex tasks
  • Multitasking support using a persistent dock, with newly introduced file-management and drag-and-drop capabilities
  • The support of “desktop-like” features that leverage the underlying machine-learning capabilities built into the new version of the operating system
Image of creating design in iPad

These enhanced capabilities will allow us to build even better, more engaging digital sales aids that further improve pharma rep detailing experiences — for docs and reps!

VR / AR for Patient Empathy, Diagnosis, Treatment & Remote Engagement

Apple has been hard at work providing a comprehensive set of developer tools to continue to blur the lines between digital and physical worlds. With newly embedded machine-learning and computer-vision capabilities supplied through ARKit — Apple’s new augmented-reality developer toolkit — Apple can support a wide range of experiences.

This release gives Apple a stepping stone in the spatial computing arena, and Apple is now in a good position to stay relevant with emerging technologies like Google’s Tango, Microsoft HoloLens, and Magic Leap slowly moving in.

Image of people playing game on iPad

With Apple working to fully support VR development on the Mac, and the release of ARKit for iOS, we have the opportunity to:

  • see an increase in VR and AR solutions that transport users into a cinematic-quality environment to experience what it’s like to live with a particular disease state, or perhaps take their mind off pain or anxiety.
  • continue to push the boundaries on how to use these technologies as supplemental tools for training, or machine learning-based visual diagnosis using the device’s camera, such as FacePrint, the app designed to diagnose Parkinson’s disease
  • help people that might not otherwise be able to attend an event like a conference or CME training participate virtually through a live-streamed 360-degree VR feed
Image of Apple VR

Getting Down to Business (Chat)

A survey by Facebook last year showed that having chat capability is more important than ever for businesses. Not to be outdone by Skype, WhatsApp or Facebook Messenger, Apple is now facilitating a direct line of communication to your brand — no social media account required! — through Apple’s iMessage app. By integrating with Safari, Maps, Spotlight and Siri, users with iPhones will be able to find you and start private conversations in an environment you can manage and stay compliant.

Image of business chat

HomePod: Digitally Enhanced Patient, Caregiver and Provider Assistant?

HomePod is Apple’s high-fidelity smart speaker that plays music from your Apple Music library and also has access to all the data Siri’s intelligence has built around your personalized profile. Much like Amazon’s Echo, HomePod can respond to the same “Hey Siri” related inquires as the Watch and Phone using an improved, more natural voice that can now be male or female.

Image of apple speaker

For now, the HomePod seems to be focused on providing a great music experience, but in the future, it may be useful for healthcare needs, including:

  • providing personalized educational content and recommendations based on a user’s previous behaviors
  • recommending care plans, and connect patients and caregivers with others in a community for support
  • enabling healthcare providers to allow a speaker to listen to — with the proper HIPAA-compliant measures in place — point-of-care discussions and “prescribe” pharma-sponsored digital content and messaging, curated by doctors for patients
  • easing the administrative burden on doctors through semi-automated medical transcription and data entry into health record systems


While each new piece of tech Apple unveiled was significant in its own right, perhaps what was most significant was the bigger picture: Apple continues to produce tools pharma marketers can use.

Whether your audience needs a personalized blend of interactivity through mobile, mHealth, Web, chatbots, voice-based assistants, or VR and AR, it’s now easier than ever to use machine learning and artificial intelligence to seamlessly create an interconnected physical and virtual web of personalized user experience.

Are you ready for the next digital transformation? Contact us today to learn how a brief, collaborative innovation workshop can start driving results for you and your organization today.