Rough notes from the sessions I attended at Google IO. The sessions are going to be available in Video so, if you want more detail (or more accuracy :-)), you can check out the videos.
Vic Gundotra Keynote:
· 2 hour session walking through entire conference material mostly with demos: Open Social, Google Web Toolkit, Android, Gears,
· 8 Main Conference Tracks with multiple concurrent sessions in each:
o AJAX & Javascript
o APIs & Tools
o Maps & Geo
o Mobile
o Social
o Code Labs
o Tech Talks
o Fireside Chat
· All recorded and will be available publically.
OpenSocial: A Standard for the Social Web
· How do we socialize objects online without creating yet another social network (there are already at least 100)?
· API for controlled exchange of Friends, Profiles, and Activities (the update stream)
· Recommends Hal Varian’s (Google Chief Economist) ”Information Rules”
o OpenSoicial is an implementation of Chapter 8
· Association of Google, MySpace, and Yahoo!
· More than 275M users of OpenSocial
· How to build an OpenSocial application?
o JavaScript Version 0.7 now and REST services coming soon
o Three groupings of the API
§ People & friends
§ Activities
§ Persistence
o Programming model is async. Send a request and set a callback function that gets called on completion.
o Update of activity field: postActivity(text) – also supports setting priority
o Example server side REST services:
§ /people/{guid}/@all: collection of all people connected to the user
§ /peple/{guid}/@friends: friends
· Main sell is to allow small sites to gain critical mass when friction of yet another login system and initial lack of users would have blocked. Make it easier on users.
· Showed a map of the world showing that different social networks have won in different geographies all over the world.
o E.g. LiveJournal (Rusia), Orkut (Brazil)
· OpenSocial gets you to all their users so plan to localize your application (OpenSocial is designed to support localication)
· OpenSocial Terms:
o Container: the site (Hi5, MySpace, etc.)
o Owner: author/owner of the page
o Viewer: person viewing the page
· Apache Shindig is an open source implementation with a goal of allowing new sites to host open social applications in well under an hour.
· Shindig is an Apache Incubator project: http://incubator.apache.org/shindig
· Summary: make the web more social, current version is 0.7, and 0.8 includes REST.
· OpenSocial has 11 sessions in addition to this one at Google IO.
Google App Engine
· This session packed. Others quite lightly filled.
· Google App Engine does one thing well
o App engine handles HTTP requests, nothing else
o Resources are scaling automatically
o Highly scalable store based on BigTable
· An application is a directory with everything underneath it
· Single file app.yaml in app root directory
o Defines app metadata
o Maps URL patterns in regex to request handlers
o Seperates static files from program fiels
· Dev Server (SDK) emulates deployment environment
· Request Handlers:
o Python script invoked as though it were a CGI script
o Environment variables give request parameters
§ PATH_INFO
§ QUERY_STRING
§ HTTP_REFERER
o Write response to stdout
· Runtime is Python only but the fact that it is specified in app.yaml suggests that more will eventually be added.
· Showed Django support and how to use GAE with Django
o Showed a minimal main.py
§ Import os from google.appengine.ext.webapp import util, ….
o Also showed minimal settings.py
· Note: Existing Django apps will NOT port easily to GAE.
Google Docs + Gears == Google Docs Offline
· Google Docs Offline Architecture:
o Document editor
o Spreadsheet editor
o Presentation editor
o Authentication
o Docs Home (doclist)
· Overall, no big breakthroughs. It’s just Docs offline but its work well done.
· Challenges to disconnected operation:
o Upgrade is a challenge: Now that code is being installed remotely, the server needs to support old code at least until the new code is pushed out and installed.
· Possible solutions for static resources: fail to upgrade, sticky sessions, resource database, or serve the old version.
· Solution implemented: resource database with a per-server cache
o Rolling upgrade for HTML: hard code the offlineVersion and request it specifically – it will fail during rolling server upgrades but the speaker argued that it wasn’t worth the cost to avoid this failure.
o Security: Decided to not do auth remotely and rely on O/S facilities (if you have access to the data at the O/S level, you get access). But they do provide support for multiple users since most power users have multiple personas (work and home at least). Multi-user support is via putting the email address of the user in a cookie. They have an loggedin and a loggedout manifest. The loggedout manifest redirects to a dialog to chose one of your existing accounts. This either sets the loggedout cookie to an appropriate email address or fails. (loggedin cookie doesn’t have an email address – it has the google security context).
· Recommendations:
o Need to provide debugging tools (online can look at server logs – need something for online)
o Rollout initially a small number
o Support disabling offline experience for a user
Under the Covers of the App Engine Datastore
· Speaker: Ryan Barrett: App engine Data Store Lead
· Bigtable in one slide:
o Scalable structured store
o Types on each value
o Single row transactions
o Two types of scans: 1) prefix (physically contiguous), 2) range scan (also physically contiguous)
· The entities table:
o Primary GAE table
o Stores all entities in all apps
o Generic and schemaless
o Row name is entity key
o Only column is serialized entity
· Entity key is based on parent entities (root to child, to child, etc.)
o Note: Can’t change a primary key but can delete and create a new entity with new key
· Queries and indexes:
· GQL: Google Query Language
o A tiny subset of SQL. Most clauses restricted. Added the Ancestor clause.
· Big table only supports scan. No sorting and no filtering.
o Because they have no knowledge of the app or data shape, they convert all queries to scans since that is all BigTable can do.
o Indexes:
§ Kind Index (kind, key) where kind is child, grandparent, parent, …
§ Single-property index (kind, name, value key) : Serves queries on a single property. (there are two indexes: ascending and descending)
§ Composite index: defined by the user in index.yaml (generated by the dev environment if you run queries over all needed composite types).
o All index comparisons are lexicographic
o They support index intersection. Multiple equals filters and an equals filter and an ancestor restriction for example (just do index anding).
· Indexes space consumption is not charged for since they don’t want to make people go to considerable pain to avoid using, for example, composite indexes. Ryan went on to explain that this is what he “wants” but it is not a committed decision.
· If a query can’t be satisfied with a range scan, they query will be failed (need index exception).
· Transaction model: all writes are transactional
o All writes are written to journal with timestamp
o No locking – they use optimistic concurrency control.
o Each entity has a last committed time. All reads access last committed time. All writes check to ensure last committed hasn’t changed. The committed timestamp is only updated after the full value is written out and the log entry is written. The log entry is a big table row and each row supports atomic writes. He didn’t provide enough detail to fully debug/understand the commit protocol implementation.
o You define entity groups (defined by the root entity – all descendents are in the same entity group. Only the root has the timestamp.
o He did say that all writes to a entity group are serialized so make the entity groups small.
Working with Google App Engine Models:
· Speaker: Rafe Kaplan
· Other object relational mapping systems:
o ActiveRecord
o Django
o Hibernate
· Does not map to an RDBMS
· No pre-existing schema
· No joins, No Aggs, & no functions
· Showed how to model relationships
James Hamilton, Windows Live Platform Services
Bldg RedW-D/2072, One Microsoft Way, Redmond, Washington, 98052
W:+1(425)703-9972 | C:+1(206)910-4692 | H:+1(206)201-1859 | JamesRH@microsoft.com
H:mvdirona.com | W:research.microsoft.com/~jamesrh | blog:http://perspectives.mvdirona.com