Thoughts on Flash on Froyo / iPhone Safari Plug-In SDK

Some initial observations regarding my first twelve hours with the Flash 10.1 beta on the Nexus One:

Flash scrapes a pass in the performance test. Videos can be choppy at first, but after a little while they seem to play smoothly enough.

Flash utterly fails the UI test; forget any video playback controls on any Flash video you plan on watching. Tested it with the Engadget Show live last night and was unable to change the volume, also tested with some Zero Punctuation reviews and similar problem. It appears that Flash renders at the Nexus One's native resolution (480x800) which means the playback bar on virtually any Flash video is so tiny it's unusable. There's no automatic fullscreen mode when you double-tap a Flash context like some of the earlier demo videos mentioned, but the 'fullscreen' button on many Flash video players work fine (of course, Flash contexts that handle double-taps themselves to go fullscreen work fine).

Photograph here with SD card for comparison; you can't pinch-zoom in fullscreen mode, so how on earth are you ever expected to press any of the controls? This is worsened by the fact that Android phones' touch sensors aren't the most accurate either.

I can't give comments on battery life one way or another. The Nexus One's battery life is poor by default, but after watching a short Flash video or two I left the phone (in sleep mode) on my desk overnight; the battery was dead by the time I woke up.

Overall, for Flash to work on mobile devices virtually every Flash video site is going to have to redesign their Flash video player to support touchscreens. They may also have to re-encode all their videos for best performance (Flash keeps popping up 'This video is not optimized for mobile').

That being said, with the web as it is today, I don't see how Apple can ignore Flash for much longer unless there's significant movement to HTML5, which offers less features and generally performs worse than Flash, in the next year. To be honest, I don't see that happening; the pace of the web is glacial, at best, even when Apple is trying to force peoples' hands. It's really nice to be able to go to a website and not have to worry about whether it uses Flash or not; before this, I'd have to get a laptop out if a site demanded it. I don't think that's where Apple wants the iPhone or iPad to be, and I think the egos on both sides have done enough damage.

The Solution

I think Apple should put together a Safari Plug-in SDK for iPhone OS, and let developers decide. It's no more difficult to create a Safari plugin for iPhone than it is for the Mac, and Apple can sandbox them as much as they like, run them in a separate process and use IOSurface to draw them in a web view. Let the plug-ins access the same APIs that App Developers can, and let any developer submit them to the App Store. Why should Adobe get special privileges? Why can't Engadget, say, write an iPhone plugin that adds additional interactivity to their website when viewed on an iPhoneOS device? Should we wait for the web to catch up, or should we make something great using native code?

To prove a point, I have sample iPhone Safari plug-in source code available from my GitHub. You can test it in the Simulator, or you can test it on a jailbroken device. Make something amazing with it, and give Apple a reason to give us a plug-in SDK.

Update

I posted a video on Vimeo showing part of Lights Off running as an iPad Safari plug-in, just to give a concrete example:

20 comments:

  1. I don't agree that flash should be run on the iPhone. Look how bad you said it ran on the nexus. It's going to take atleast a year for it to be fully worked out and to be able to run smoothly on any mobile phone. And apple will not put anything in beta on their products.
    ReplyDelete
  2. What I don't understand though, is why doesn't Adobe simply release Flash as a plugin for jailbroken devices? There's nothing stopping them from doing this, and like you showed in your sample plugin, the framework for plugins is already there.
    ReplyDelete
  3. @Mohammad To clarify, Adobe have said that the current beta of Flash for Android doesn't use hardware acceleration yet. Performance will improve. It didn't run bad on the Nexus One at all, it's the UI and battery bit that was bad.

    @Henri Watson They technically could, but it's not the kind of thing a 'big'/mature company like Adobe would do.
    ReplyDelete
  4. Flash video runs poorly, yes. The rest of flash runs nicely though. I can go to websites that don't force me into a mobile version that have flash and actually use their website now. Like Mohammad said, this is beta. Although I don't know how much they can fix in terms of UI this late in the development process.
    ReplyDelete
  5. From my own experiences, I would prefer to keep flash away from mobile devices. It's not the most reliable on a desktop computer, and with no force quit options available on many smart phones, I'd rather not have to keep restarting.

    The point I found most interesting here, was that you said that many flash sites and their content will need to be re-done in order to be suitable for mobile. If they need to change things around for flash to work best on mobile, wouldn't it be just as easy to recode in HTML5?
    ReplyDelete
  6. Ooooppppsss, can't see any video, just white space :-(
    ReplyDelete
  7. "I think Apple should put together a Safari Plug-in SDK for iPhone OS, and let developers decide." This is the bit I have an issue with. Nearly all the developers that have voiced an opinion over section 3.3.1 and/or Flash on the iPhone platform have all said the same thing; "let developers decide"--as an aside if anyone actually really has the final say, it the consumer. To think that a company that develops and sells a closed platform, as _any_ company is entitled to do, should let 3rd party developers decide the fate of their product is naive. The iPhone OS is a closed source platform, and I think that the majority are forgetting this salient point and like it or not, it's Apples ball, so they get to make the rules. Android on the other hand is open, different rules apply. It's not for developers to decide, that was the whole point of Jobs' missive ; been there, done that, didn't work out well for us.
    ReplyDelete
  8. @Simon but I'm not saying let the developers decide the future of the platform; Objective-C Safari plugins would have the same APIs and sandbox that apps have. It's the same stuff, just rendered in a web browser. 3.3.1 has nothing to do with this, nor am I against 3.3.1
    ReplyDelete
  9. http://bit.ly/94EtQ0

    really? this performs worse than flash would? lets remember here that Apple is pushing for the latest and greatest browser tech in their devices, with deep level access to 3d acceleration.

    that said, install the latest webkit nightly:
    http://bit.ly/aDTDi5
    and look at this:
    http://bit.ly/bN6LRa

    then try to tell me that we still need flash. because i really gotta say the writing is on the wall.
    ReplyDelete
  10. @Drew that's not the argument; there are a few key things that HTML5/CSS3 do nicely hardware accelerated, but others blow up on iPad or iPhone because of the CPU speed (for example, tap on the display at www.steventroughtonsmith.com on an iPad or iPhone and notice how jerky the animation is - that's a single CSS3 property, should be super fast).

    To get the complexity of a real Flash app using Canvas/CSS3/HTML5 you're looking at similar CPU usage and battery drain for both, if not less for Flash (because Flash has years of optimization more than HTML5, even if it is still a hog).

    My main point is to show a use case for native Safari plugins on the iPad/iPhone; there are some really cool things that could be done (and safely) because of the walled-garden approach.
    ReplyDelete
  11. I think a lot of People act is if HTML5 is head and tails above Flash when they are both not that far apart. HTML5 is more reliable and faster in sometimes, Flash is more reliable and faster in others.

    As far as Video, it should be noted that current mobile devices DO HAVE HARDWARE ACCELERATION FOR HTML5 H.264. Flash on Froyo does not have it just yet.

    To me, the reason Jobs and Apple are pushing H.264 vs Flash is more because of Financial Reasons than Technical concerns. After all, Apple is a member of MPEG LA and would receive royalites if everyone used HTML5 with H.264.
    ReplyDelete
  12. @ Ed Parry there is a force quit option on the iphone. You hold down the power button till you get a screen change (the off option) then you hold down the home button only for about 5 seconds and it will force quit the app that is running.
    ReplyDelete
  13. Can someone please help me with this??? I tried to create a window based project in xcode but it wont work im using the iphone 3.0 simulator---please help ive been tackling this for about 3 hours. Please compile a project and upload it somewere?? I just want to test the flash plugin--my email is spicysquirrel@gmail.com. Please someone reply
    ReplyDelete
  14. How to test this on Jailbroken iPhone? I have put all files in /System/Library/Internet Plugins but is not working... I have try reboot and respring device.. OS 3.0 iPhone 3G. Can someone help me?

    Thanks in advance!
    ReplyDelete
  15. Same as Retoocs has. Someone help us!
    ReplyDelete
  16. @shayan @Retoocs @Ádám this is for developers, not for users. There's nothing here to use, it's sample code. You need to compile it for device, codesign it, and install it in the directory, then reboot Safari
    ReplyDelete
  17. Hi,

    Your project is interesting, but, if i understood we need a Developer Account to use this plugin? Without we can't codesign it and we can't compile it?

    (Sorry for my English, but i'm French)
    ReplyDelete
  18. True it would be great to have a compiled and codesigned version for us user dummy's to try rather than letting only the devs have all the fun Steven :D
    ReplyDelete
  19. Hi Steven,

    I downloaded your source code plug-in and installed it on the iPhone simulator, but it doesn't seem to work.

    What am I doing wrong?

    I created a directory named "Internet Plugins" on my mac under "/Library/Application Support/iPhone Simulator/3.2/Library" and copied the .webplugin to it.
    I then tried surfing to a site that uses flash, but didn't see the red dummy view indication.
    I tried naming the directory "Internet Plug-Ins" but that didn't seem to help either.

    Can you advise?
    Thanks!
    ReplyDelete
  20. I cannot find the Settings->Safari->Plug-ins On-off toggle option? I've seen screenshots of it on the net and on forums being discussed, but my phone does not have it. 3GS-ios4.
    Any idea how I can get that option in my safari settings to be able to turn plug-ins on or off?
    Thanks
    ReplyDelete

Subscribe to: Post Comments (Atom)