Dev Meme

Russ Fustino and I recently presented the first three episodes of our popular It's All About the Tools! (It's AATT!) seminar series in recorded MSDN WebCasts. Joe Healy joined us for two episodes, and I ended up on vacation with my family during the 3rd session. All-in-all, they worked out really well, and we got great feedback from the live audiences that joined us during the events. We'll hopefully get Episode 4 under our belts sometime in the near future.

Below are links to my previous posts on the content of the sessions, and links to the recorded sessions. Note that the sessions weren't recorded exactly as laid out in those posts, but that's part of the fun of doing live sessions. For example, in Episode 1 we were having VPC problems, and I had to skip the IE Developer Toolbar (which we caught up with in Episode 2). At the time I was unhappy because that's one of my favorite tools, and an anchor of Episode 1. It all worked out well in the end, I'm happy to say.

• Episode 1 tools links and recorded session (event id 1032379990) - presented on June 16, 2008
• Episode 2 tools links and recorded session (event id 1032380036) - presented on June 19, 2008
• Episode 3 tools links and recorded session (event id 1032380038) - presented on June 30, 2008

Updated 2008.07.16: oh yeah, and the slide decks for the WebCasts with embedded links to everything are on my website (all zipped pptx).

Please let us know what you think of these talks, and give us any suggestions you have for other tools topics.

You're probably thinking - what in the world is a Tweener (hint: it has nothing to do with 12-year olds)? The Florida developer community is fortunate to have been offered use of the Orlando Convention Center the weekend between TechEd 2008 sessions. The Tweener is a FREE all-Florida CodeCamp-style event that will include 2 full days of sessions including Russ' Toolshed It's All About the Tools! episodes 1-4, SQL Saturday, .NET CodeCamp, Day of Agile, ITPro Camp, MS Universities (intensive training on DotNetNuke, .NET, OCS, SQL, VSTS), cert exams, Hands-on-Labs, and more.

Whether you go to TechEd or not, you won't want to miss this jam-packed weekend, put on by the Florida developer community. If that lineup isn't enough incentive, there will of course be a killer party Saturday night, so you'll want to stay over both days. Here's the official Tweener website where you can see the agenda and register.

I'm teaming with Russ Fustino for a continuation of our It's All About the Tools! seminar series. We're reprising AATT! Episode 1 and AATT! Episode 2, and adding brand new Episodes 3 and 4. The special thing about these sessions is that they'll be recorded live for a TV show we're working on. Here's the Tweener link to register for the AATT! track.

Our blurb introducing the sessions goes like this:

Did you ever feel inundated with developer tools? Did you ever feel you're missing something that could make your life easier as a developer and a power user? There are so many tools available to choose from and it keeps getting tougher to keep up with them. Tool familiarity continues to be one of the top developer challenges.

This series of seminars is all about tools for developers, by developers. Join Microsoft Florida Developer Evangelist Russ Fustino & Microsoft MVP Stan Schultes and others as they bring you demos and discussion about their favorite developer tools.

Here's the tool list and links for Episode 3:

• Expression Encoder and Silverlight Live
• Virtual PC
• dev.live.com/quickapps
• XML Notepad 2007
• Reflector & Addins - .NET decompiler and more
• PowerToys Pack Installer for Visual Studio 2008
• CodeKeep - VS add-in for managing code snippets
• SyncToy / FolderShare - file sync/backup tools

And the tool list and links for Episode 4:

• Vista Ultimate Backup and Restore
• DotNetNuke portal and the IWeb web services module
• Resourcer - create VS resource files
• Regulator - regular expressions test tool
• LinqPad - Linq query execution utility
• IE7 Pro add-in
• IE8 Developer Tools
• StickyNotes add-in for VS 2008
• Expression Design
• FaceBook Applications API

As before, we don't cover every tool in every session - mixing things up keeps it more interesting for us :)  You can register for our AATT! Tweener events here.

DotNetNuke is a really cool piece of software, and it's pretty easy to install and operate. The hardest thing about running a DNN site is upgrading it. This is especially true when the data and content "rescued" from a shut-down site aren't able to be staged on a development server for upgrading.

In the process of changing web hosts for my SarasotaDev.net site, I'd retrieved database and content from the old site, and installed an upgraded DNN site on a new host with a temporary calendar page. You can read my previous post about the situation and the basics of that process.

Now it was time to take stock of the content I'd been handed from the old site, and map out a plan to get the full site up on the new host. The site also was due an overhaul, so what really had to come out of the upgrade process was the membership database. Everything else would be resurrected content in a new skin and with a new layout.

In a nutshell, the backup database retrieved from the old site wasn't whole, so the site wouldn't come up on v3.0.12 and run in a staging environment. The SarasotaDev data was plundered out of a SQL database that was shared on the host with other web apps. I got an .exe file created with Red Gate's SQL Packager which restored what appeared to be a complete DNN installation, except that it wouldn't run (I tried!).

I didn't get very far in troubleshooting before I realized that many, but not all, of the foreign keys between tables were garbled up. This basically rendered the layout and content unusable, but the membership data seemed to be relatively intact. A few (turns out there were 7) users were dropped in the translation. But that was a whole lot better than asking every user to re-register.

Over the past five years I've gotten quite a bit of experience creating, managing and upgrading DNN sites. It's interesting to note that every upgrade I've done has been different. So, I've developed a few rules that I follow when doing upgrades and migrations:

• only migrate database contents between the same version databases
• upgrade at most one major version at a time
• always use a clean web.config from the target version, carefully match machine keys
• change only one thing at a time
• test & backup!!
• strive to end up with a clean database and directories

I feel that last bullet is an important one. During an upgrade, I make a serious effort to end up with a clean database and directory structure. Repeated upgrades of DNN sites cause some entropy to occur, which can make the site less stable over time. Call it DNN rot (a phenomenon similar to Windows rot).

I use Microsoft Virtual PC for my staging environments, as I can maintain clean development systems without conflicting technologies which add variables to the migration process. Earlier versions of DNN require different .NET Framework and SQL database versions than the current versions - so having a way to isolate problems is very helpful. Especially so when the data needs work, as in my case.

The process took many iterations to determine the most direct line in which to retrieve the membership data and upgrade it to v4.8.0 as installed on the live site. Below I've outlined the final process I followed, and why I chose that particular path. It's very important to note that your mileage may vary. You need to stage and test your particular situation, as I'm positive that it's different from mine and any other DNN upgrade scenario. Each site and upgrade is unique.

Following is the "path of least resistance" that I found for my upgrade from DNN v3.0.12 to v4.8.0 (the site was originally running on .NET Framework v1.1 and SQL Server 2000 in a hosted environment):

• before taking the old site down - I exported module contents, took screenshots, grabbed a database backup and directory contents, and noted the version of all 3rd party modules (there were none).
• I restored the SQL package as my source database, and cleaned out all data I didn't want to migrate (this installation had more than one DNN portal, and I only wanted to keep the main portal contents)
• installed a clean staging portal (DNN v3.0.12 - matching the original database), made the machine (encryption) keys the same, and configured the site without any roles
• deleted both the DNN and ASP.NET membership tables from the target DB, following this table order: Users, Roles, Profiles, UserRoles, UserPortals, then aspnet_Applications, _Users, _Membership, _Profile, _Roles, and finally _UsersInRoles (NOTE: I believe the ASP.NET membership tables aren't needed now, but I haven't tested this myself).
• imported membership data from the source DB, allowing identity inserts, following the reverse table order as when deleting data (to ensure no key constraints were violated)
• tested logins and assigned roles to the content, backed up the site (DB and directories - always save web.config)
• upgrade v3.0.12 -> v3.3.7, test & backup (only move one major version at a time)
• change FW v1.1 -> v2.0, test & backup (only change one thing at a time)
• detach DB, attach to SQL 2005, test & backup (SQL 2005 wouldn't work for me on FW v1.1)
• upgrade v3.3.7 -> v4.8.0, test & backup (do you see a theme here?)
• created a clean install of v4.8.0, configured it as the live site, test & backup (goal is to end up in a clean DB and directories)
• migrate the membership data to the clean DB (following the table order above), test & backup (migrate data only between the same DB versions)
• finished host & site configuration, test & backup
• deploy to live site (copy & restore database backup)
• migrate content from old site, upgrade to latest version, deploy new skin (future steps)

At the end of the process I also had to go through and fix up user role assignments as some of the table pointers were apparently garbled in the original SQL package. My main concern here was the Subscribers role, as it's what I use to send the newsletter. FYI - when the newsletter fails to deliver to an address (due to people changing jobs or providers), I simply remove them from the Subscribers role to un-subscribe them.

I could also see that some users were dropped by comparing the DNN membership data with the ASP.NET membership tables, which are managed in parallel. For the 7 users that were dropped, I simply sent them an email explaining the problem and asked them to re-register. That was out of 180+ names in the DB, so it wasn't too bad overall.

One other factor mitigated a lot of upgrade problems: I didn't use any third party modules on the site. These can be a problem if support has ended, an upgrade path isn't available, or issues occur during migration. Using standard modules tends to make an upgrade go easier, as the DNN core team is managing module changes.

Since I was doing the migration in my "spare" time, it took nearly 2 weeks of evening work to sort out the process I ended up using to migrate my site data. It wouldn't have been nearly as time-consuming if I'd had a SQL backup that worked from the beginning, but sometimes you have to work with what you've got. Plus, it helps keep me in practice!

Recently, my SarasotaDev.net group lost the hosted website that a friend had provided for free since we put our DotNetNuke v3 site up, back in Apr 2005. Like when someone tosses you a hot potato, the challenge is to keep from getting burned as the usual string of real-world problems plays out.

I needed to get the site back up quickly because I'd just sent the monthly newsletter highlighting our upcoming meeting and I expected some site activity. Here's how the just-in-time migration process went down, right before Andrew Connell came to Sarasota for a SharePoint dev session (which was great!).

I knew about the shutdown in advance, but true to form I delayed action until I had just a few hours left. I'd given the process lots of thought, so I had a pretty good idea what to do - all I really needed was a SQL database backup, and the contents of the Portals directory.

The hosting account was needed for another business purpose, but this isn't really a problem because as an MVP I can get a free hosting account at ORCS Web (put them on your short list). I had the imminent time of execution pushed back so I could send out the monthly newsletter to the current membership, and grab some of the current content.

I was able to retrieve the following three items from the existing site, just before it went down:

• SQL Server database backup "package"
• copies of the web.config and /portals directory
• HTML contents of the current meeting schedule from the old homepage

It's that word "package" there on the database backup that got me. Upgrading a DNN site is all about staging the upgrade and rehearsing the steps. It's pretty straightforward as long as you have a SQL backup and matching content. Piece of cake :P  Except the SQL package I got wasn't whole - I couldn't just restore it and run.

The hosting site had a shared SQL database, so the SarasotaDev data was extracted from it and an executable file was created with Red Gate's SQL Packager (and I didn't do the extracting). Let's just say that some foreign keys were messed up and some users were dropped. Other than that, I was good to go.

The first order of business was getting the SarasotaDev.net domain answering on the new host, and a basic homepage up with the current meeting and events schedule. All the domain and account setup magic went pretty well. I made my first OrcsWeb hosting request around noon, and had a working site in about 2 hours on a new IP address with an 'upgrading' page and answering on the right domain name.

Next I needed a working DNN site with basic content. After setting up the account, uploading the DNN v4.8.0 Install package, and configuring the database connection, I got the site up with the calendar as a temporary page. A little bit of site configuration and it was already getting public hits at about 3 hours elapsed time.

I tried staging up the retrieved database and content, but quickly came to the conclusion that some brain salad surgery was going to be involved. So I decided to relax a bit and do the upgrade right. Given that the database wasn't whole, going from the old v3.0 DNN site to v4.8 would require planning, offline staging, and probably many iterations. 

Hurrying through the process is a recipe for nothing but trouble, so I didn't. The good news is the process is straightforward if you plan and test carefully. I'll talk about the gory details in my next post.

This past weekend found over 600 developers at the fourth-annual CodeCamp in South Florida (Miramar) USA. This was another class event, with SFL CodeCamp leader Dave Noderer and his crew putting on a superb show in a great facility on the campus of DeVry University (kudos to Ed Hill for hosting us again!). I put some photos up on my gallery for your amusement.

Besides the 600 plus attendees, there were almost 70 total sessions spread among no fewer than 12 tracks of 6 sessions each! Tracks included: Intro, Web Dev, Smart Client, SQL/BI, Agile/CI, Architecture, New & Cool, ASP.NET MVC, SharePoint, and Silverlight. Another full track was done in Spanish.

I had so many choices for topics to present, but I came up with these two sessions (plus a fill-in):

• Web Service Software Factory Modeling Edition - this software factory from Microsoft Patterns & Practices group is used to build the server side of a web services infrastructure. The new item is the Modeling Edition, which records your answers to the wizards and replays them at codegen time. Great stuff. My slides can be downloaded here.
• It's All About the Tools! Episode 2 - this great developer tools oriented session was a joint presentation with Russ Fustino, Microsoft Developer Evangelist. Russ and I have a lot of fun with the AATT! sessions, and this series may one day become a classic (we wish).
• Russ and I did It's AATT! Episode 1 as a fill-in session, which was full even though we'd already done it 3 times in the South Florida area this winter.

After the CodeCamp came the PubClub, arguably the best part of the weekend. This edition was held at JP Mulligans in Pembroke Pines, an Irish pub with a ringside seat at an ice hockey arena! Then we were off to the Hard Rock and finally to bed. Sunday was a race home across Alligator Alley and up to Sarasota to see the Super Bowl in the late afternoon.

All-in-all it was another great developer community weekend! I'm looking forward to the other 2008 Florida CodeCamps including Orlando on March 22, Tampa in July, Jacksonville in August, and Tallahassee in September.