Authoring contextually aware Apps

As the Internet of Things provides more and more powerful, compelling objects to program — it will become increasingly important to have tools for non-programmers to control and publish content to these intelligent, low-powered devices, kiosks, sensors and mobile handsets.

In fact these tools will not be programming tools at all. They will be authoring tools designed to build contextually aware, personalized Apps that will know WHO you are, WHAT time it is and what date, WHERE you are, WHEN it’s appropriate to include others and WHY you should care about this App — in the first place.

The key thing is that these tools will be utilized by Authors, not Programmers — as programming is too far removed and complex for most creative, intelligent authors to master.

With BigData changing the landscape and mainstream “artificial intelligence” systems finally appearing it’s now possible to build Apps that only a few years ago seemed like science fiction.

Yet to build these kind of Apps today one would need: a data scientist, a machine-to-machine learning expert, a mobile developer (or two), API experts, UI/UX designers and a whole bunch of senior level coders to connect to the Cloud and implement abstract notions such as:

“If you’re going shopping THEN make sure to remind me WHEN there are specials at THIS shopping mall which are ALSO on my shopping list .

These kind of “Minority Report” solutions are a marketers dream come true and promise to drive much of our ‘on-line’ commerce of the future. But what ELSE can be done with the Internet of Things?

Shouldn't we be building contextually aware personalized Apps that are MORE than just shopping?

What’s needed are authoring tools that non-programmers can utilize that can put the Internet of Things to use —in the right context for this Age of Context — for education, health, entertainment and all sorts of uses — which we can’t even imagine today!

Authoring rather than programming

For “normal” people to play and have fun and fail and try again — authoring tools are needed that allow Authors to iterate and experiment. It is far too expensive, complex and involved for any single person to do that today.

Groups of people must be engaged to develop mobile Apps which talk to devices and the Internet, which will bring to bear BigData and Intelligent based experiences. But marshaling and managing groups of people costs lots of money and increases the complexity of synchronizing creativity with engineering and deployment.

Authoring intelligent contextually aware Apps involves working with industry conventions — such as people and events and maps and media— and integrating those conventions with two kinds of “intelligence authoring” tool paradigms; one for foreground activities and the other for background smarts.

Activities are the actions that end-users interact with their Apps and any social interaction that these experiences may include.

Activities include displaying text or photos or playing videos or audio/music — on a smartphone or tablet.

Authors would create these activities by placing buttons on the App screen for end-users to make choices, take surveys, record audio, interfaces that allow end-users to select, discover or express themselves.This is what’s referred to as “interactive activities.”

Smarts are what happens when your App monitors your movement, activity, input or status. Smarts are pre-built sentences which express a specific kind of “logic” that gets executed at the same time that a whole bunch of OTHER Smarts are executing.

Introducing Interface

Interface is the name of a new authoring tool environment being developed — as we speak.

Interface Authors can create both foreground activities and background smarts with constraint-based tools — based upon what content, settings, social graph, venues, schedule, configuration, web sites, Apps, devices and context — the end-user provides.

These activities and smarts are encapsulated along with customized content and output as Agents — which then playback inside of an Interface App.

Interface in one sense — is a personalized publishing system enabling Authors to create one-off Apps — for any one specific end-user. This enables professionals, counselors, trainers, tour guides, personal assistants, shopping or travel agents, consultants and prosumers — to run their own business producing contextually aware Apps for THEIR customers.

Compared to today’s blogging tools (such as Wordpress or Tumblr) a contextually aware authoring tool would display content — not just in one place or one website — but across an entire distributed network of ‘end-points’ — which could include other mobile handsets running the same App.

Contextually aware Apps could also interface to and talk to other mobile Apps — and plug into IFTTT and Yo.

Or the App could refer to web-based eCommerce sites, Wikipedia, social networks or messaging systems.

Notifications, content feeds, marketplaces and media aggregators will all be part of the distributed network that is created when contextually aware Apps are deployed.

One key feature in the authoring tools that create these new kinds of Apps is that they will provide some way of storing end-user’s content, data or media — and making that content available as part of the distributed network. That means authors can create “tools” themselves which integrate social interaction into the experience.

The trick will be to provide the right templates which represent the social interaction “motifs” or “themes” that people will find fun and compelling.

If an Author wishes to build a contextually aware custom App for their customer, groups or friends, the first thing they’d ask their customer is: “who are your friends, colleagues and service agents who will you be interacting with?”

Social Interaction and the IoT

People are stored in Facebook, LinkedIn and Twitter nowadays so importing one’s Facebook social graph is a nice way to start thinking about “easy-to-use” authoring tools.

All one would have to do is point at their Facebook or Twitter account and create a sub-set of people whom they wish to interact with. If some poor soul is NOT on Facebook or LinkedIn, then by-gosh then we’ll figure out a way to talk to them as well. Do they have an email account?

Building contextually aware Apps begins by creating a list of people who I interact with and what happens when I encounter these people. All of the people who come by my house, who make deliveries to my home or office, who I have to interact with are exactly the people that I want my authoring tool to have “logic” about.

I want my home to open it’s doors to the FEDEX guy or plumber. I want my friends to be greeted with a salutation and some special “inside joke” when we encounter each other. I want my car to also be “intelligent” and display pictures of my friend and I together — as they get into the passenger’s seat.

These kind of contextually aware App follow end-users — through their day through — waking up at home, commuting to work and interacting with people all day long. These contextually aware Apps have to understand one’s daily schedule, when dinner is served and what needs to happen before they go to bed.

Contextually aware Apps will watch over your house and you’ll have one dedicated to watching over YOU and YOUR life. Corporations, Professionals and Government will produce contextually aware Apps which will know who you are and what you need. Commerce and marketing Apps will will know exactly what you want, what your shopping list has on it and what you reward yourself — when the time is right.

Ideally contextually aware Apps would manage your finances, health and wellness — but we know how that goes — right? Even the best advice is sometimes ignored.

So when do we pay attention and utilize our contextually aware Apps?

And when do contextually aware Apps become a nuisance and they get turned off?

Well the answer is in the hearts and minds of 1,000,000's of contextually aware App builders — like yourselves!

Interface authoring tool environment

Interface believes that for the “Internet of Things” revolution to truly take off we need to empower non-programmers to have at their disposal — all the tricks, techniques, services and capabilities — that “the big boys do!”

Location and Geo information are also foundational concepts for building contextually aware Apps. Here the notion of “placing pins” on a Map has long been an honored authoring paradigm. Certainly creating a “zone” on a map is something any intelligent Author could do.

Now the trick is simply to define what happens when YOU travel inside this zone. What information or media is associated with this pin on a map at this exact location? What tasks or activities do you wish the end-user to do — WITHIN this zone or Venue? That should be easy enough to do — right?

The trick of designing easy-to-use authoring tools is defining ‘ constraints’ into the tool, so that ONLY a set list of possibilities is capable of being selected and “authored.” If we can pull that off, then we can free our Authors from the shackles of programming code.

Authoring tools enable Authors to select and choose from a constrained set of possibilities. The goal is to provide a wide enough range of expression, tasks, services and interactivity — that end-users will care about.

Authoring tool systems need to “ingest” these possibilities first — and then display them as choices in a drop-down menu or grid of tiles.

The Interface authoring tool environment features a ‘Context Map’ that serves as the mechanism to import and ingest relevant information, profile settings, lists of venues, locations and events, etc.

The Context Map also serves as a ‘data visualization’ mechanism showing what data is changing — at what time.

By being able to visualize what data will change — when — Interface Authors can design interactive “activities” that App users can then interact with — at the right time.

Events are another important convention that most people understand. Calendars are the traditional way that people interact with Events. Events are usually correlated with venues or specific places so a list of venues is also something that an authoring tool will want to have built in as a ‘convention.’

Time, location, venues, people all are the cornerstones of building contextually aware Apps. And media and content are the filler.

Authoring tools for contextually aware Apps should have a means of uploading and storing media — as a standard feature. Media tools simply point to cloud locations or files — whether they be internal to the Interface tool or stored in Flickr or YouTube.

Custom pages and images can be created or uploaded as the tool is designed to provide authors the capabilities of building “knowledge bases” of media information that can then be published and deployed in the Interface App. Authored experiences can then be created that include ANY kind of media or text information in a timely and appropriate manner.

Persistent storage will be built into the Interface authoring tool as a function —that enables Authors to create social interaction experiences where the end-user of the App can take photos or shoot video — that can then be stored as part of the App.

Providing persistent storage (sometimes refereed to as CRUD) baked into a tool and intermixing it with people, events, venues and maps provides a wide palette of possibilities from which to build contextually aware Apps.

One way to think about the Apps being created by Interface is to understand that there will be a Interface marketplace — where Interface Apps created by authors will be bought and sold. Since these Apps will tie into external web services, content and people — the marketplace itself will connect to other marketplaces and parts of the distributed network.

The Interface fitness app


As we've explained in this video — Interface’s first App framework will be an App designed to work in conjunction with a Personal Trainer.

Take a look at the video — it’s only 02:00 long!

Interface authors can be any kind of professional, consultant, counselor, trainer, educator or marketing person. We started with Personal trainers because fitness devices are the current leading personification of the Internet of Things today. 2.3m fitness units were sold in Q1 2014.

Personal Trainers would utilize the Interface authoring tool to define the exercise routines they wish their client to workout to — and attach instructional videos, photos or audio explaining their suggested technique or regimen to those routines.

Personal Trainers are very often asked to provide relevant information customized to their client’s needs — such as a woman who just had her baby and wishes to lose that extra baby fat. Other clients may need Diabetes related info or ask for the latest G-BOMBS recipes that the Trainer (now acting as a nutritionist) may offer.

The Interface authoring tool environment is a personalized publishing platform enabling Trainers to create the exact information their client require —and include that information inside of the customized Apps that Interface produces.

Trainers can encapsulate their “style” and essence of training in a smart App that has been explicitly designed with individual workout clients.

Trainers would also be able to record their voice and suggest music that could be triggered based upon logic applied to the real-time data coming off of the client’s exercise device.

Interface believes that this sort of ‘smart’ audio playback will produce compelling and powerful experiences and enable authors to create their own “Zombies, Runs!”

App end-users would be able to access the Interface Context Map to visualize which aspects of their digital lives were changing.

This Context Map is the same interface found in the Interface authoring tool environment.

Interface App end-users would also be able to monitor all of the ‘smarts’ going on simultaneously inside the Interface App.

These ‘smart’ monitoring tasks are personified as ‘sentences’ expressing the logic’s task.

Interface Apps will be able to hold MORE THAN ONE Interface Agent — at the same time.

Please refer to an earlier white paper on the Interface Agent framework.

Authors create Apps

Lets go over the steps required for building a contextually aware App:

  1. Ingest all of the relevant information and data into the tool. This includes end-user’s data and device configurations, relevant groups of people, venues, events and location based information. It also includes what Apps the end-user currently utilizes and what content feeds and channels they subscribe to.
  2. Define foreground “activities” — which is a range of possibilities that the App’s end-user can interact with eg. “what does the Author wishes their end-users to DO?”
  3. Instance background “smarts” — which entails choosing a pre-built sentence ‘template’ and customizing that sentence with the appropriate nouns, verbs, adjective and prepositions desired to be monitored and operated on by the App’s background tasks. Based upon the criteria configured— the App would then automatically (or with user prompts) provide relevant, timely changes, updates, notifications, content or other kinds of IoT’s “tasks” and “things.”
  4. Iterate and tweak these settings and designs — by emulating the App. This would give the author immediate feedback as to what they just authored. This is a key aspect of any powerful authoring tool environment.
  5. Deploy the App. And maintain/upgrade the App.

Distributed Networks of contextually aware Apps

The final piece of this equation is connecting these contextually aware Apps to other Apps, web and mobile based services, streaming on-line media and the Internet in general.

Most people using Apps today have OTHER Apps they’re already using as well.

For an authoring tool environment to become successful it must work with others and support OTHER platforms, Apps, social media and infrastructure.

For Interface Apps to react to context means that these Apps must be aware of other contexts going on around it. What data, activity and people being interacted with — is relevant contextual data that all feeds into an end-user’s global context status.

Interface contextually aware Apps will enable the Author’s end-users to stay “jacked-in” and relevant.

Its one thing to come up with a compelling social experience. Its another to demand that all of the participants in the experience MUST SIGN-UP and download a compatible App!

That has to end!

Compelling contextually aware social interaction will ONLY happen when different Apps, created by different vendors enable users to interact, publish, message, notify, friend and in general — interact with each other in a distributed network manner.

IFTTT provides an excellent middleware layer over which our distributed networks can inter-connect. Interface plans on leveraging IFTTT in many ways as our mechanism for providing inter-connected contextually aware experiences over distributed networks.

The Interface authoring tool will be a channel in IFTTT and produce Apps which will plug into IFTTT.

We can’t really call the distributed network that Interface Apps interact with “our” distributed network. Because it’s all of our network — glued together by infrastructure services, such as IFTTT.

Interface Apps will collect the same performance data that comes off of one’s Fitbit or Health monitoring devices. So Interface Apps will work in conjunction with Runkeeper, MyFitnessPal, Runtastic, Strava or Moves.

Interface Apps will talk to the same SmartHome devices that everyone else does — but do OTHER kinds of fun and educational activities with them!

Interface tools will know what YOU want to author and control, WHO you want your App to interact WITH, WHEN important times, dates and events happen and what kind of content you wish to display. Exactly when.

This will enable new kinds of contextually aware experiences that will display THIS and THEN if THIS happens — take what has been stored in the Apps storage bins and display THAT information. In a nice little tile pattern.

The marketplace is the conversation (as the Cluetrain Manifesto proclaimed) and the buying and selling of Interface Apps will give rise to an entire ecosystem of cottage industries — each offering their own domain expertise packaged up and available for purchase or lease.

Interface agencies will appear that specialize in creating custom Agent frameworks and provide support and training to OTHER Interface authors.

For rich, engaging and compelling contextually aware Apps to flourish in an open distributed network, Authors will require more than just “trigger-action” kind of gestures.

Interface’s ‘smarts’ editor utilizes full sentences (not just “trigger-action” sentence fragments) to express logic and intention connected to the base conventions of the tool.

If a range of potential venues needs to get plugged into a smart sentence (that chooses between what time of day, what group of friends or what notification needs to get sent,) then the Interface sentence editor’s “mad-lib” drop-down menu would appear RIGHT ON the noun, verb, preposition or adjective.

Here’s an example showing a range of performance results that an Author would choose from — when authoring a ‘smart’ audio playback monitoring task.

Constrained based authoring tool environments can express virtually ANY kind of content, media or message — at exactly the time, place and context — that the Authors WISHES to express. The Author simply has to “author” the content and interactive experience and then tie it in (attach it) to some smart task or gestural social expression.

This means Interface authors can create Apps that can work on conditional branching, conjecture or “what-ifs” without being programmers. Logically sequenced social activity can be “authored” into any sort of game, educational lesson or research study.

We believe that the best results from Interface will come from the people most unlikely to create them — school kids, junior marketing assistants, interns, advocates and senior citizen center managers.

Timeliness, convenience, appropriate behavior and powerful relevance are the characteristics of “intelligent” contextually aware Apps.

Its possible for normal people to author and control these kind of aspects and produce these kind of results. But what they need are the right tools to do that —because hiring programmers and building an entire company to do just one idea— is just not gonna happen.

This is what Interface is all about.