Mozilla Trippin

Standard

Last week I took 3 days of vacation to make a trip down to the Mozilla office and work on a few projects. My trip went very well — it gave me time to focus and whiteboard things with Chase.

my desk

I spent the majority of my time working on AUS – which could stand for Automated Update Service or Application Update Service. I’m not entirely clear on which one it is, so let’s just say AUS. I was able to update the AUS Lite code to reflect our needs for branch updates.

In the previous version of AUS, branch testers would turn on their automatic update settings in Firefox or Thunderbird and then receive an update notification. The successive update would then upgrade their installation to the Aviary Trunk. Not good.

So the updated code now has awareness of the Version -> Branch relationship, which is a fancy way of saying that it knows which versions get what updates.

In the first run-through, AUS only knew about major updates for an entire product. This was to test the AUS client GUI in the beginning phases of the Deer Park feature, which is one of the major things that will be added to 1.5 applications (both Firefox and Thunderbird).

So now branch testers can stay on the branch, and everyone’s happy. Upcoming features for AUS would include a better build-system bootstrap, and a better way to generate XML output into static files (as opposed to being semi-dynamic). The upcoming feature list is relatively short, but it warranted some great discussion between Chase and I.

Needless to say, the future of Mozilla’s software update looks very bright indeed.

Other things I worked on:

  • Automated test scripts for AUS to verify correct output for pre-planned test cases that are stored in an .ini file. This ensures that new builds pass a sanity check, and would be ran from a web-interface or command line.
  • Bouncer updates, particularly with the user interface for the build system bootstrap via SUM file using Lars’ awesome loader.py script.
  • Some brief discussions with Rafael about the future of addons.mozilla.org (AMO).

Overall, I got a lot accomplished in the two days at the Mozilla office. Thanks to Chase and Karen for setting up this trip.

Unlimited snacks and redbull can make for a very productive geek.

Some Thoughts About Mozilla Update

Standard

Despite what some people are saying about Mozilla Update, the Mozilla Foundation has focused the right amount of energy in the right direction.

Before focusing fully on AMO, a concentrated effort had to be made to upgrade their Software Update architecture and user interface (AUS). Critical/security updates to the application core take precedence over extensions in any update system, and the Mozilla Foundation is no exception.

Great strides have been made towards the next version of AUS, and the rewrite of AMO v2.0 has been well underway.

The first version of AMO has been plagued by poor performance, UI difficulties and lack of robustness. What the project lacked in the very beginning was a technical lead that understood how to make a scalable web service. That was not there because Mozilla Update in many ways was an afterthought in the wake of the success of Firefox 1.0. It barely had its head above water, covered in the whitewash of the 1.0 wave.

Now that the smoke has cleared, Mozilla Update (AMO) is seen for what it is – a nice try.

Was the Mozilla Foundation wrong in letting the community release early and often? No. I feel this is just a necessary first step in the right direction, and I hope that in the midst of v2.0 and all the bickering and complaining, everyone involved in v1.0 at least learned from the experience and might understand what to watch out for this time around.

For me, it’s been a wild ride, and I look forward to the completion of v2.0. There is a lot of work left, but I’m going to work hard to do my part.

Stop worrying about who to blame, just fix the problem.

AMO Framework

Standard

Most of the template and db framework is in place for AMO v2.0. I struggled a bit with how to relate Smarty and PEAR::DB objects to the rest of the modules. After a lot of tinkering, I ended up deciding to not instantiate new ones with every module, but rather use global instances of these objects and pass them by reference in the constructors of each module.

The drawback? Not as pretty. The reason? Speed and scalability. As it stands, the framework behaves like this:

  1. Define config variables.
  2. Instantiate Smarty object.
  3. Instantiate PEAR::DB wrapper class.
  4. Attempt to connect to database, display “gone fishing” page on failure.
  5. Filter inputs into $clean array (if any).
  6. Populate $sql array with properly escaped values for use in queries (if any).
  7. Query for data using global $db object.
  8. Pass all necessary data to Smarty via assign().
  9. Disconnect and destroy $db, display page output.

With some more poking around, I was able to use auto prepend/append to group all init and finish items into one file. As a result, each PHP document was successfully limited to the following steps:

  1. Filter inputs into $clean array (if any).
  2. Populate $sql array with properly escaped values for use in queries (if any).
  3. Query for data using global $db object.
  4. Pass all necessary data to Smarty via assign().

See an example of this at work.

Pending updates to the main template, we could be moving straight into reworking the developers area by next week. That will present some added twists, like session handling, RDF parsing and complex items like the review queue — but it’ll be fun.

Until then, the oh-so-important public side of AMO will improve one piece at a time by hammering on this framework.

The smallest deed outweights the greatest of intentions.

Bridging the Gap

Standard

There has been some discussion about the direction of Mozilla’s Update Services and where they will be going as Aviary 1.1 approaches.

There are three tiers:

  • Addons (extensions and themes)
  • PFS – Plug-ins and the “finder service” that helps you find correct plug-ins based on mime-types
  • AUS – Critical updates that your app checks for periodically (that red thing on the upper right)

Tonight, I thought about what makes Mozilla and open source unique. What sells it to the community, and makes people like you and me — once aware of the option — gravitate towards open source alternatives? What made Firefox successful?

Building off Kveton’s assessment, community interaction and feedback has led to direct results that are visible in record times. From improvements in nightly builds, minor revisions, update services, etc. — users have gained a sense of ownership and sense of community when working with certain projects or applications. There is less of a gap between developers and the public demand that drives them.

Software engineering in the private sector, driven by corporations, can tend to rely more on focus groups, customer surveys and error reporting tools. Microsoft, based on these sources can work to improve products to ensure market share and customer satisfaction.

So, while I was talking to Thompson in the car about it, I came up with the point that although both sources are legitimate, only one has a sense of ownership and community that is tied directly to and supported by the founding organization. Microsoft, for example, would not alter the IE trunk to correct standards interpretations for years despite mounds of feedback. There was no turnaround there… and it wasn’t the first time.

But I don’t like geting into the MoFo vs. Microsoft game. It’s not really a fair comparison because Microsoft has much larger problems caused by their enourmous user base. Regardless, in projects like Firefox or Thunderbird there is such a close relationship between developers and end-users that the turnaround time for bug fixes and application improvements is remarkable and unprecedented. I have not seen such a connection in the Microsoft community.

Surely, though, it won’t be smooth sailing forever. As time progresses, and the population of MoFo’s end-users increases, they will face some of Microsoft’s problems:

  • Scalable update architechture
  • Progressively difficult regression testing

The community will likely survive its growth, but there are some things we should start doing now to prepare for the future. One of the best ways to help prevent these growing pains is to invest more time and effort towards ensuring that the gap between end-users and developers never widens.

And Mozilla is not without direction. This is already being done by tools like reporter, which was recently added to nightly builds. Sites like Mozillazine, SpreadFirefox and Bugzilla also contribute to opening paths of communication between the developer and their users.

Keep in mind that it doesn’t stop there. We have a responsibility to do more for users than make their applications user friendly. We should give them the option to participate, to feel the sense of ownership and community that makes these apps special. What better than augmenting the update infrastructure with more user-facing forums, an improved rating system, and upgrading critical update options and reliability?

Aviary 1.1 is already moving towards an improved critical update mechanism that is focused on smaller patches, more options, and a “set it and forget it” mentality somewhat similar to the hands-free Windows Update services you’ve seen in Windows XP. Some people might think that’s it — no, it’s just the beginning.

Much like Windows Update, critical updates dealing with security or serious flaws in application architecture will mostly just be blindly installed. “Yeah, yeah, just do it so I’m up to date.” It’ll be like Symantec’s virus definitions. You hardly know they are updating themselves. And that’s great — that is what I’ve been hoping for in Mozilla applications.

Now users don’t have to worry about critical updates or security patches. They don’t have to constantly install new binaries, and as applications mature, critical updates will hopefully taper off. Now they can worry about having fun, playing with new tools and innovations that extend an already great application base.

So give them an easy way to browse, install, update, troubleshoot and discuss extensions or themes. Done properly, a reworked and revamped addons site could provide these venues. It would ensure and improve the sense of ownership and community unique to all Mozilla applications.

Forward.

Standard

UMO was re-released a week ago, and we have been happy with its comeback. Despite some minor usability issues, things have gone over pretty well, considering the codebase.

It was good to see such a flurry of activity; from a revitalized sense of excitement in chatrooms to the corresponding boom in submitted extensions. It shows how much addons mean to Firefox and Thunderbird.

UMO v2.0 sits on the horizon — a re-developed architecture, built for scalability and extensibility by an experienced core. It will offer all of the things lacking in v1.0. Our goals will not have changed, and we will strive to answer all of the great feedback we’ve been receiving.

And meanwhile, Lars has been cranking away at his modifications to Bouncer v2.0, which will be out very soon, pending some final changes regarding file input/additions.

It has been a very busy April. I haven’t had much time to stop and write. But in some ways that’s a good thing.

May will be another step in the right direction.

Quantum Leap

Standard

Scott Bakula could have summed it up with, “Oh boy…”. Everyday an entire industry leaps from point to point, making great strides towards an uncertain future. We see glimpses of what is to come, but are unsure of what it will really be. Cloudiness marks the path of technology. When we get there, it seems so obvious, but for so long it all seems so terrifying and uncertain.

The familiarity of where we just were lingers as we are thrust into the next step in the evolution of technology. Very few foresee where we will go tomorrow. Those who do, as cliche as it sounds, use it for good or for evil.

And sometimes, we prepare as a community for what will happen. As information sharing and collaborative software development evolves, so does our awareness of technology’s own evoloution. Two — or thousands of — heads are better than one. As communities have been empowered by new tools, they have driven some exciting projects.

Apache, Mozilla, Debian, Gentoo — oh, and Linux itself — are all fine examples of how a collective effort has paved the way for technology before the way was really known. More than anything, they have provided the foundation for the next best thing.

Soon our software will be alive. It will evolve before our very eyes. It will learn how to cope with new viruses, spyware, spam or increasing demand for particular features. It will catalog your mistakes, helping you get what you need with greater speed, clarity and precision.

Gone are the days of the 8-floppy install suite. Welcome are the times of the 4 megabyte installer with one hand firmly grasping the internet. Welcome is the client-side checkbox named, Always know what the hell is going on and let me know.

Web-based application update services will have a growth spurt in the next two years. It started with net installs, Windows Update or Symantec virus definitions. It ends up with a community-based effort to combine a next-generation appplication toolkit, innovative and scalable web update services, and distributed mirror management.

With all the talk about where projects like Mozilla have been, we are once again looking backwards, with fear and uncertainty about where we are going. We generate this unrest because we don’t see instant gratification. We don’t get our king-sized serving of technological fries whenever we want it.

And yes, sometimes these things take a bit of time. It’ll take more than 5 minutes at the Burger King drive-thru to make this all work; much longer. In many cases it takes much longer than the private industry would find to be economically viable. But it will happen, and more importantly, it will happen the right way.

Because we’ve come too far to pack our shit up and go home in defeat. We’ve found ourselves on the brink of changing history. We have an opportunity at hand, as a community, to reclaim control of the presentation of information, and to safeguard it against all possible threats. Think about it.

Never before have we had the chance to make information truly free. Even then, freedom was a lost concept, a mere construct formed by those who were trying to market it. Now think of having complete control over all of your inputs. What a beautiful yet simple concept.

What we’ve failed to realize is that we control our own destiny. As a community we can reach our Atlantis, and we control where we leap to, just as Sam Beckett found out in his last adventure. And to blow up the metaphor, once we collectively figure this out — instead of stopping, we’ll continue to leap with a newfound awareness; uncertain of where we’ll end up, but definitely going there on purpose and with a clue.

What role will you play in the evolution of technology?

Mitchell on the Firefox Release

Standard

Mitchell Baker recently published a great recount of what happened during the Firefox 1.0 launch. It’s a good read if you have the time.

Like I’ve said before, it was a great community effort and it was encouraging to see everyone working together despite the divisiveness of our time. There is a lot of hard work to come, though.

Stop to smell the flowers, but don’t smell them so much that you get high and forget to keep going.

Well, Maybe Not…

Standard

So maybe Peter Torr’s claims address some things that will never be resolved. Ultimately, you will never be able to fully trust anything. There is always some chink in the armor – which is why nobody ever guarantees that anything will be 100% secure.

Torr’s blog posts basically create fear, uncertainty and doubt about all binaries in general. But what is complete bullshit is that he pinpoints firefox as the source of this problem, when in fact he is merely questioning software distribution as a whole – which is something Microsoft has struggled with and still has not solved.

In the end, security is just an idea. It is even more a feeling than an idea. The sense of security is what gives consumers confidence in a product. The truth is that in most cases a reasonable sense of security is all anyone ever wants – true security is almost unattainable. You are always vulnerable to something.

To some, that is an alarming thing. But when you look at the definition of vulnerable, you begin to realize that the only way to be truly safe is to not be open. And, yes, in a way that philosophy is in direct conflict with the nature of the web and the nature of open source development.

Microsoft can safely assume that security means closing all doors, since that is what their business philosophy pretty much encourages. “Close all doors and capitalize on the bottleneck” would probably be their philosophy. Not only do they want you to be scared, they want you to pay to be safe. There is a lot of money to be made there.

I think the correct approach to security with software is the same as in real life. Use common sense, and when that isn’t enough make efforts to educate yourself. Don’t leave your keys in the car. Don’t leave your doors unlocked. Don’t trust strangers.

Of note is the fact that in real life most severe crimes are caused by someone you know. This is because trust opens you to harm. When discussing a central signing agency like Verisign, etc. you have to consider that if you empower a central point of trust it becomes a central point of failure. If you trust Verisign to handle all of your stuff, you become ignorant, and it becomes likely that something will fly in under the Verisign blanket and hurt you.

None of that means you have to live your electronic lives in fear of everything out there. Just be safe, man. Keep informed, don’t download random shit, don’t trust sites you aren’t familiar with, etc.

A part of that, ironically, is not trusting Microsoft, which is something Peter surely doesn’t mention in his article. Not using IE has been a great way to secure your computer. Not using Outlook Express is a great way to avoid complications with mail. Not using XP is the best way to avoid damage caused by viruses, etc.

Overall, you will never be safe, but you can do things to decrease the probability of being “attacked”. If you follow common backup procedures, then the worst case scenario is that you lose a night of reformatting your system drive. Surely it isn’t worth living in fear of the unknown for that.

Security is just a feeling, and if you accept that you are on the road to being secure.

Motivation

Standard

Peter Torr’s blog post on signing Mozilla binaries (Firefox, primarily) was a good article. Most people will flame it, but his argument holds some water. I respect his opinions and recognize the validity of his statements, especially considering his follow-up post, which responded to many of his Slashdot readers.

Overall, this is good stuff, it helps Mozilla become stronger in the long run and gives us additional motivation (there was already plenty) to “do the right thing”.

In the coming weeks we will be working on increasing the security of Bouncer in its coming version (v2.0) in addition to many other features (versioning, languages, statistics, statistical exports).

More to come, folks, doing the right thing takes time, and sometimes a little bit of pressure and criticism.

Bouncer v2.0

Standard

v2.0 is almost complete. This past week I worked on completing the additions of languages and versions, and also made quite a few adjustments in order to accomodate the new database and its relationships. v2.0 should hit production within the next week (pending testing).

Thunderbird 1.0 came out this week, which was relatively lackluster in the wake of the Firefox 1.0 release. Polvi agreed that the lack of excitement is partially a result of market share and competition. Mail clients are largely web-based, and those that aren’t are pretty thorough and do not posess many of the security flaws or standard-ignorance that IE posesses in the browser market. Regardless, you should get your hands on it and test it out. By the way, bouncer is dealing out Thunderbird 1.0 — http://download.mozilla.org/?product=thunderbird&os=win&lang=en-US. 🙂

Nonetheless, Thunderbird is a great program, and beats the crap out of Outlook Express. That said, it’ll be interesting to see what happens with the mail client world — will it start to merge with the web or maintain its tradition of being relatively stand-alone? Time will tell.

One interesting point is that with the development of Gecko and XUL, why did Mozilla try to develop a stand-alone mail client? Isnt’ the idea behind the revolution the un-desktopilization (holy crap – new word!) of the internet and its applications? The future of the desktop lies in the browser and its integration with modules that expand on HTTP — I think developing a complex XUL-based extension to Firefox might have been in line with what some see as the next generation of desktop applications.

My thought is that in order to get consumer buy-in for Thunderbird and increased interest in the foundation, Mozilla needed to cater to the existing market, which means creating replacement tools that are much better in the ways of security and functionality. When, though, will we collectively take the plunge towards a more seamless web/mail browsing environment? The next ten years will be very interesting.

All things considered, Thunderbird is an excellent application and I use it exclusively. Mutt fanatics and Outlook junkies aside, it is probably the best solution around if you’re looking for a newsgroup/mail client that is not retarded. The development team did a great job.

Thunderbird isn’t circa 2040, but as a desktop app, it kicks ass in 2004-05.

reclaim your inbox.