I am attending the Fujitsu Labs of America Technology Symposium again, and today’s topic is one very dear to me: how clouds and ‘Ambient Intelligence’ can be distributed to transform the human experience.  Here is a summary of a couple of talks.

Contextual Intelligence as a Driver of Services Innovation

Teresa Lunt, Hiro Kishimoto, Tomm Aldridge, Wayne Ward.

Teresa Lunt: (PARC) Ambient Intelligence is called by PARC Contextual Intelligence.  It is all about sensing and recording in the life of a worker. For example, the Digital Nurse Assistant tracks things in the environment of a nurse to help them do their jobs better.  Sentiment Analysis and topic Detection in Social Media can be used to gather information relevant to the worker.   Such techniques have been used to help detect fraud, waste and abuse in health insurance, transportation, credit cards, social services sectors.  Call centers can mine audio and email transcripts for learning how they are best handling situations, and to support people to get better.  Real time analysis and intervention is the ultimate promise.  One example was a productivity solution in the car that senses your situation, like being stuck in traffic, and automatically informs the other meeting members that you will be late, or initiates a change of schedule.

Kishimoto: (Fujitsu) His group in Japan is taking on the challenge extending cloud to M2M.  Human Centric Intelligent Society is Fujitsu vision.  The current cloud needs to be extended to support sensors. This is called Machine to Machine (M2M) network.  He claims that this network is very different from the network we have today: e.g. number of sensors is very large, limitation in process capacity of the devices, issues with limited power, not always on, along with security and privacy issues.

Tomm Aldridge: (Intel Labs Europe)  Set out the goal to help discover how to make the world more energy efficient.  What is an electric society?  One that allows living in a more energy efficient way.  There is a huge migration to megacitites, and there will be many cities over 20 million people all over the planet.  There will need to be a lot of infrastructure to support this, and the opportunities expand exponentially.  How do we manage combustion free zones in the middle of a city?  He is taking a fresh look at how people use energy.  People will make better decisions and will develop better futures when enabled with information about their lives.

Ambient intelligence platform vision: the opportunity to define a well structured platform from the sensors themselves, to the services, to the end user.  While we can draw a stack diagram just like anyone else, but user centric design, meeting the needs of the user, and appropriate technology, is what will make it succeed or fail.  Data security, and machine trust must be woven in across all layers.  Dr. Eve Schuller has been studying trusted cloud research, and how to assign attributes to low level devices to let you know how trusted a device can be.  Challenge is to figure out how to do resource optimization.

One project is simply the Personal Office Energy Manager (POEM) which is now a product.  It is a USB device that measures ambient light, temperature, and humidity.  This can provide feedback to the building manager on how well employees are being accomodated.  Have built a sensor hub that can aggregate this, store it in a more traditional data structure, like a SQL database.  Produces a calendar of energy usage, and specific goals set and tracked.  It use the metaphor of a flower wilting to indicate success or failure to the user.  Lots of analytics on trends.  There are even ways for the workers to control the energy usage (turn things on and off) directly from the screen.

The big frontier is maintaining security and trust in these sensor networks.

Wayne Ward: (Sprint Nextel) Machine to machine is defined as connections between everything that is not a cell phone.  Everybody in this room probably has about 8 connected devices today.  John Chambers was quoted as saying there is trillions of dollars of market potential.  Usage based insurance will completely change auto insurance, where you actually pay for the actual miles you drive.  Gave an idea of a vending machine that has a camera to measure traffic, and then detect the age and gender of the people walking by, and possibly dynamically advertising to those potential customers.  Fitness monitoring, health monitoring, to help you manage your lifestyle.

Q: Everyone is positioning to “own the data”.  How are networks evolving to accommodate this traffic. Will congestion get worse or better?
A: One network won’t satisfy all needs.

Q: How is the ecosystem evolving to make an open environment so that network suppliers can cooperate and make the standards.
A: Single platform is better.

Q: How will information be controlled in this new network?
A: Maybe this idea that you own your own data and take it with you is a dinosaur idea.  The new generation allow the distribution of tons of data about themselves; the older generation is not comfortable with this.  What is the eventual pattern be?
A: Europe is very cautious about data, but America allows data to be freely moved.

Q: Question of Privacy.  Young people are more inclined to publish their data on the internet.  What about the case where a company measures something about a customer that is not in the line of the business, and shares it.  For example, what if the device measuring power discovers that the computer is idle for several hours during the day, and the employee gets fired because they are supposed to be working at that time?
A: to be honest we have not really studied that interesting case.  The paying customer is the one we have mostly studied.  You can opt out in many cases.  It would be very hard in labor laws to use that data against an employee.  This is an important topic.

Q: What about open platforms for taking sensor data
A: Tomm: think it is wonderful that there is innovation going on on the edge with what are called “motes”.  A lot of this work is coming from people who are not the traditional developers of these things.  Used to work with Berkeley, and now we come full circle to see open source driving a lot of new innovation.

Q: We are talking about senses, but this information is including txt and email messages.  How do we broaden this sense of a sensor?
A: Teresa: Built a leisure activity recommendation system that reads text messages while discussing with friends.  Use that along with location, time of day, past history, in order to offer suggestions and to predict their activities that they might want to do.  There is also the opportunity to unify communications and instrument them to add to a more complete understanding of what people are doing.  Interfaces to other systems: like salesforce, or maintenance support systems, etc.  There are a lot of opportunities to merge this all together.

Solutions in Healthcare, Automotive and Smart Cities that Leverage Sensor Networks and M2M.

Jason Tester: (The institute for the future) Focus on three things: Creators, Context, and Computation.

1. Creators:  Creators are like DIY makers, hackers, open hardware.  Does anyone have a “twine” sensor?  Temperature, orientation, and motion.  Started on kickstarter, it is a very low cost simple sensor.  DIY example of twine: motion detection, turn on a web cam.  Or make a phone call.  One was: if the closet with Christmas gifts was opened, send a message to gradpa.  Another interesting one is a “FitBit” to measure activity.  There is gamification, but this user did not find enough interest.  This user wired it so that if he does not move enough, it cuts the power to the fridge, causing food to spoil.  This provides the motivation to keep moving.  Another is “air quality egg” which is a mass produced air quality sensor based on arduino.  This plugs into a map showing where the air is good or bad around the world.  This is a kind of crowd based approach which may go counter to the official.

2. Context: Context is how technology is being used, what needs is it meeting.  Home, workplace, and cities.  2008 was a landmark year as more people lived in cities than outside.  Also where more people had mobile phones than other.  Urban crucibles: people with problems meet people who can solve them.  Pulse-point application for iPhone that let CPR trained people report their location to the emergency services, and they could be called in to help the fire department.  So far: nobody has failed to respond when called.

3. Computation: three eras: abundant data is the first.  The next is the internet of things.  What is past that: the age of networked matter.  Everything has the ability to talk to other devices in the network.  Moving from a world of spaghetti connections where everything has the ability to talk to everything.  But future will be much more locally connected.  Sense-making that can happen across different scales.  Not just Capable of intuition.

Mark Zdeblick: (Proteus Digital Health) Digital technology that will bring better care to more people at lower cost.  Analogy is communications.  200 years ago if we wanted to have a conversation we would get together, or we would send a letter.  The post office is a centralized approach to communications.  Then came telegraph, telephone, and internet which is now completely decentralized and costs only 1000 emails for 10 cents.  How can healthcare be like that?  Right now healthcare is a centralized mode – you go to the center where all the doctors are located.  How can this be changed to be more like communications.

Proteus put chips inside medicine which communicates when you take the tablet.  The data stream includes heart-rate, accelerometer, and we can determine how much sleep someone is getting and how healthy they are.  Example: two women with cancer both want to help each other be successful in taking recovery medicine.  One way is to sign up people with a team that help each other, with a possible prize for the best working team.  Parkinsons is related to gait, and so measuring gait can be an indicator of how well something is doing.  Only data allows you to deliver a nudge that is not a nag.

Gave a description of a ingestible event sensor.  The fluids in the stomach become the electrolyte for the battery.  Communicates to a wearable receiver.

Brad Templeton: (Singularity University) He writes and blogs about autonomous cars which he calls “robocars”.  Why is this important?  Human drivers are pretty terrible.  We kill about 33K people every year in the USA and 1.2 million worldwide.  The cost of accidents to American economy is $230 billion, or about 8 cents per mile.   93% of accidents are due to human error.  40% involve drinking.  We spend 41 hours of life every year to traffic jams.  25% of energy use goes to cars.

However, computers drive cars now much better.  Not full AI, but just the intelligence of a horse. They don’t see like a human, they don’t drive like a human, but watch 360 degrees constantly monitoring everything.  In the next few years we will see simple self-driving cars.  Cadillac is coming out with a “super cruise control” that handles the steering wheel.  Audi, WV, Mercedes are coming out with cars. Changing cars will be the biggest impact on the world.  How does it work?  Lidar on top, not really a camera, senses everything.  There is also radar which can see through fog.  There is a GPS, but not really used for driving, instead just a rough, global idea of where you are.  He played a movie showing the Google self-driving cars navigating through various difficult situations.

Mobility on demand is where you pick up a phone, and a car appears when you need it.  No need for a garage.  This changes your buying decision for you particular hour to hour needs.  Most trips are alone and across town, and a dinky car would do for that.  This will enable electric cars, because you don’t care about the range of a taxi – just whether it gets you to where you are going this trip.  Used to think that the lawyers and politicians would be the barrier, but actually laws are moving quickly.  Particularly for disabled, or drinkers.  This might cause more and longer trips, causing more sprawl.  Biggest barrier will be fear of being killed by robots.  Moore’s law comes to transportation.  Early adopters are stupid people with too much cash.  Bottom up approach.  These vehicles don’t park, and so we don’t need parking lots.  Turn the parking lots into parks land.  And then there is pizza: the one product must be able to be delivered anywhere within 30 minutes.  Time for an Apollo like project to get a car that can drive a businessman to lunch, and return him back to work safely.

Jay Primus: (S.F. Municipal Transportation Agency) Working to get parking back on track.  Challenge is to get people to available spaces quickly.  Signs that lead people quickly off the streets.  Can modify prices up or down 25 cents every 4 to 6 weeks.  This allows tuning of rates to fit the demand of a particular area.  A big focus on making it easy to pay, so that people avoid tickets more regularly.  Started with a parking space census, a count of every parking spot in the city, and they believe they are the first city in the world to do this.  All the usage data is collected and processed to optimize parking rates – and they realized that Excel was not going to cut it – so they invested in some significant analytic techniques to clean and structure the data.

Q: How to encourage more of this kind of innovation
A: The best innovation is bottom up.  The other side we need to think about is the privacy issues.  One important thing about the SF project is that there was a real sensitivity to this.  This is counter intuitive, because many project would wonder why you should “throw away data” but it is important that you are sensitive to only collect the data that you need, and avoid collecting things that are not needed.

Q: How does it cost per pill for the ingestible sensor?
A: 20,000 sensors on one wafer for $1000 – so about 5 cents each is a ballpark figure of costs.  They will go down when this gets into mass production.  The value could be far larger than this.  There are business models where you give the sensors away, and you are only compensated when the pill is taken.  The information collected can be more valuable than the sensor.

Q: How elastic is the price on parking?
A: This is one of the biggest questions.  This may be the first time that such data is available.  Looking like about .3: for every 10% change in rate, the demand goes up or down 3%.

Q: What sorts of failure modes and rates do you see?  Parking and Driving.
A: there are 8000 spaces with sensors in them.  Several hundred per month.
A: Robots don’t drink, except on Futurama.  The biggest failure mode for humans is drinking.  For robocars there will certainly be other failure modes yet to be discovered, but so far there is more fear than reality.

Q: Can we include the idea of “forgetting” some of this data.  We seem to collect things forever, but is it good?
A: There are data retention limits in some domains.  There are also new data retention minimums so that law enforcement can retrieve old things.

Mapping the Brain

Lester Russel, Miyoung Chun, Tom Dean

There are two points to cover here: First is the machine brain interface.    Showed a picture created by Heide Pfutzer who is not able to move any muscles at all, but did it was a direct neural interface. The second area is understanding the brain and modeling it.  There is an uncomfortableness to this topic.  However, there is some potential to use ICT to help understand the brain.  There are a quadrillion synapses (10^15).  That is just the number of synapses, but the number of combinations is a much much larger number.  We need a multi-disciplinary approach.

Miyoung Chun: The next frontier of scientific discovery will come from combining multiple disciplines together.  The Brain Initiative resulted from 12 key events in four categories.  First category are subject discussions, second is funding discussions.

September 2011 workshop at the interface of neuroscience and nanoscience.  Then came the blue-sky session.  The holy grail problem was “we don’t really understand how the brain works.”  We know how to look at brain activity at single cell level, up to about 200 cells.  But the brain has so much more.  Each neuron interacts with possibly 1000 other cells.  This would be like trying to understand a movie by watching one pixel.  Then we wrote a concept to the while house.  Many meetings followed with many funding agencies.  Wrote up a manifesto published in Neuron and later in Science.

One application is “deep brain stimulation” and she played a short film showing how this helps a patient.  There is significant surgery, and maybe this project can reduce the size of the device.  Also shows a stroke victim you had been paralyzed and used a robotic arm to get a drink.  The current technology requires a large stack of computers as well as a large probe on the head.  Hopefully this could be reduced in size to the that of a smart phone, and it might be able to stimulate the nerves actually in the arms instead of a large robotic arm.  Obama announced the brain initiative in the state of the union address.

Tom Dean: (Google)  We really don’t know very much about neurons and how they behave together.  Getting a real picture is hard.  Show examples of MRI,  CAT, and Array Tomography Proteomics.  Zebrafish are transparent, so you can see their brains working, and pickled mouse brains to see structure.  What about the future:  Moore’s law and miniaturization predict we can make a 10K transitior chip about the size of a human cell, as a probe to see what the structure is.  Power would have to come down, and toxicity as well, but real problem is getting the information out.  Synthetic biology is a possible approach that will work in shorter term.  Can make machines that are really big molecules.  Catalogs are being constructed already that list molecules that perform particular computational functions.  For example retro-viruses to trace circuits in the brain.  A more ingenious approach is to use DNA polymerase to record the electric potential at points in the neuron.  Course at Stanford: CS 379C.

Q: There was controversy around this project when it was announced.  Human genome was aggressive enough, but this seems open ended.  What is success?

A: Discussed many times.  No specific number of neurons to be measured.   We don’t really know how many need to be measured.  That is how primitive the knowledge is today.  It took 60 years to go from measuring one neuron to measuring 200 of them.  Broad opinion was that we could never get to 1 million cells.  However many others said that you really never know what advanced might come about.

Q: Contrast with Henry Markrams work, and do you plan to build simulations of neuro systems that match brain performance.
A: We did not know previously of European work, but now there is a good possibility for cooperation.  They need good data sets for their simulations.

Q: The early days of genome project they started with smaller genomes.  Is the same thing happening with the brain?
A: For example, worms have 302 neurons, why not do that?  The zebrafish has 100K neurons, and they were able to observer around 80K of them.  So there are a variety of approaches.  In the example earlier, the chip was monitoring 96 neurons, and that helped the life of patients tremendously.  So there are large opportunities.  NIH and NSF have possibly different motivations for funding for research.
A: There has been a lot of studies of mouse brains.  There are studies of doing correlations across maps.

Q: The DNA transcriptase approach to measure electric potential is very cool, but is there any work being done to synchronize the recording at different points.
A: Yes, this has been worried about.  There are some ways to send a signal to synchronize transcriptions across the brain.  However this is far from being solved.

Q:You all mentioned multidisciplinary approach and sharing results.  How should this be published.  A fiend wrote a paper citing Shannon Weaver, and the review committee demanded that the paper remove the reference because they had not read it.  As a result the paper was never published.  How can we get these papers published?
A: There are many different disciplines that need to come togher.  It is difficult.  There are more and more teams crossing boundaries.  This is a challenge, but a solution like this is required, because the problem is do hard.
A: You are right.  This is still an issue.  The tenure situation and so many hurdles.  “Science pogresses one funeral at a time.” (ouch)

Andy Bechtolsheim –Innovation is the Never Ending Search for Better Solutions.

Top three innovations of silicon valley:   semiconductors, telecommunications, and open source software.

Semiconductors:  Moore’s law is the best and most accurate prediction in history, and it is still going.  Over 40 years this is an improvement in transistor density by 1000000.  What o do with all these transistors:  more cores, of course.  Number of cores will continue to rise.  In 2020 we should have a terabit per die – this will be the amount you have in your cell phone.  Predictions that it is getting more difficult are overrated.

Networking: actually took quite a bit longer.  1965 was the original memo proposing a packet network.  1969 was the first operation, and 4 more years for TCP/IP.  Took more than 20 years to first commercial use.  Ethernet invented 1973.  1982 was first commercial version.  13 years for 100MB, 4 years for GBit.  Fiber communications grew similarly fast.  Spending on cloud servers is growing tremendously.  By 2020 the vast majority of applications will be in the cloud (90%).   For consumers a lot is already there.  But enterprise is a little more difficult, but workers would still like that.  Browser went from concept (1990) to company 1994 to Netscape IPO (1995) in a few years.  Browser did not have a search function.  1995 had AltaVista.  But in 1998 Google arrived.  Next unsolved problem, how do you find people?  Facebook was pretty late, but now dominant.  In all these the time from idea to adoption can be incredibly short: only a few years.

What can people do to accelerate innovation?  One idea is to quite job, and create a startup.   One advantage is that you can start fresh without having to worry about anything in the past.  But limited resources.  Takes a long time to build product distribution models.  Venture capital is an important part.  Year 2000 was a peak year in venture capital investment.  Number one investment is software.  What about large companies?  Three options (1) increase R&D spending, (2) collect more idea, (3) brain storming.  But Apple spends the least amount on R&D than anyone else.  They just simplify, simplify, simplify.  Jobs says “The hardest thing is to say no.”  There is little indication that increased R&D spending is correlated with success.  Second idea is market research.  Well known that Apple does not do any market research.  What really happened at Apple is they took ownership of innovation.  Internally driven, then say no to everything else.  But Apple is unique.  Still, any company can simplify.   Where do these ideas come from?  Best source is current employees.  Second and third are partners and customers.

Why is this so difficult?  Brainstorming comes first to discover.  Later you design and deliver.  Brainstorming is free, but design get expensive, and delivery is very expensive.  But management attention is on delivery.  Very little attention spent on the left.  Then the finance department demands that you demonstrate ROI on every invention.  Pretty sure that Apple does not do this.  However, without innovative products you can not win in the market.  There is a horizon effect.  A chess playing computer that looks 6 moves ahead, if there is a better move it can’t see it.  People only set goals that they can see, and never try to go beyond it.  First problem with horizon effect for public companies is the quarter to quarter reporting problem.  Most US companies find it impossible to think about things more than 12 months out.  Then the resource horizon is that if you don’t have the resources, don’t start.  Particularly if this is viewed as risky.  So all the funding oes to the “safe bet” “low risk” products.  None of the innovative products are funded.  You need to focus on priorities that are beyond the current year.  You do need a mixed portfolio: risky and non-risky.

Why do new ideas fail?  Too early, too late, too difficult, not relevant, too expensive.  Those are the five reasons.  However, the risk of not having innovative products is an even greater risk.

Examples of being different:  Apple is a company that almost went bankrupt when it had one product.  App store is amazing: 50 billion downloads.  Second example Google: started with search but expanded into maps, mail, news, earth, etc.  Prefer to hire right out of school.  20% innovation time. Impossible goals, but they deliver.  Third is Amazon: started as biggest bookstore.  Expanded to being a  cloud vendor.  Innovation from the top, leave lots of ideas percolating, and see what works.  Hire people who like “rapid rate of change”.

Q: Most important ingredient to innovation?
A: It starts from the top.  Top management must pay more attention to the early stages of innovation.  It is the quarter to quarter thinking that kills things.

Q: It is not the first to market, but the one that comes are the right time.  How to get the right ideas at the right time?
A: this is a real challenge.  There is a herd effect on Sand Hill Road.

Q: What is the most important quality of a leader?
A: someone has to decide what funding Is going to go to the right projects.  People who have this built in can be quite a bit more effective than those who have to integrate from a team or committee.