Meet the Nightly Observing List Generator

The new Nightly Observing List Generator finally exists outside of my imagination and has had it's tires kicked around by the test team. It is exhilarating and a bit scary to bring a new feature to life, particularly when I've been thinking about it for a long time. There's just no telling how it's going to work out ahead of time. It's sort of like having children. You do your best and see what happens as they take on a life of their own.

Sometimes things go badly. It may turn out that all the pieces aren't yet in place to really make it work, or that some unexpected development has left the whole idea lame and unworkable. Sometimes I just can't make it all come together to my satisfaction.

But most of the time a new idea soars. I try my best to let it do that, allowing for major changes and redesigns right up to the last minute. Having a sound design going in is very important, but I've found that the best tools evolve in ways that I can't imagine ahead of time. I see it as my job to let them do so. I simply try to hang on for the ride and when we land see where it all ended up. And here we are...


All-in-all I'm very happy with it and the test team is all thumbs up! Most of the concerns I had going in have been worked around to my satisfaction and there were some pretty cool developments along the way. One of my favorites is in the Shallow Sky auto-generated list. Among other things it looks for any fast-moving minor planets that may be visible that night. Another really cool one is in the Interesting Stars feature. As part of its function it looks for eclipsing variable stars with deep eclipses and only lists those that are predicted to be in eclipse that night. Very cool.

Another thing I really like is the ability to prune the list to a small set of randomly selected objects. I enjoy knowing that every time I create an auto-generated list of NGC/IC objects it will give me a unique set of targets for that night. There are so many galaxies and clusters available in my 18-inch. It's like closing my eyes and putting my finger on the atlas to pick my targets. Who knows what I'll find there. Yet with this tool I can be assured that the object will be a reasonably good target that is appropriate for my telescope.

Perhaps the most useful auto-generated list is the one that spawned the idea in the first place: the Showpieces listing. I've already used it myself when I had to be at a star party with very little time to prepare. Recommended targets included the setting moon, Mars, comet 17P Holmes, several nice double stars, and a good assortment of deep sky objects of various types. Planning doesn't get any easier than that!

The Last Piece

For some time now we've been testing/developing the new Database Power Search on a tab-by-tab basis. Last week I released the new Comets search tab to the test group, and the week before that it was the new Asteroid search. Together they should provide a very interesting tool for mining the databases. For instance, you can filter by asteroid type, such as "Aten" or "Apollo" or "Damacloid." Classifying asteroids and Trans Neptunian Objects by the orbit type is one of the cooler things I've added under the hood, and it's been fun trying to make the most if it. The comet search will be useful for finding out which comet your Grandma saw in the summer of '32 or to look for comets with similar orbits. But of course paramount is always the intelligent selection of observing targets. To aid with that I have added a new Visual Detectability filter and for double stars there is a new Splittability filter. These filters rely on the models I developed to predict the visibility of an object in the eyepiece, and how difficult it will be to split a given pair. Visual observers can use these filters to find "difficult pairs" or "easy" deep sky objects based on their inputs such as date, location and instrument.

At first glance the new DPS looks a lot like the old one, but it soon becomes apparent that it's been supersized. I've tried to make small improvements across the board that together add up to something a lot more powerful.

With the DPS done except for a few bug fixes, over the weekend I took up the last piece of the Standard Edition of ST3. I originally called it the Observing List Generator then decided it was an Observing List Wizard. The thing has been evolving so quickly that naming it was a waste of time. It's been part of the Database Power Search, a separate tool, and as late as last night I was considering an entire new tab on the planner for it!

One way or another by the end of this week it is going to exist outside of my imagination. I think you will all like it. Basically it allows you to quickly create a list of objects to observe on a specific night. The gold standard all along has been that it's 4:00 PM and there's a public star party tonight and you are in charge of having something to show people. "SkyTools, help! I'm in a hurry. Just give me a list!" It does more than that, but this particular application has been the driving force behind the tool. Wish me luck!

Now a word about where we are at: after this last tool is released to the testers I'll still have a lot of loose ends to tie up. There are some bugs that still need fixing and a lot of little things that need smoothing over or to be brought up to par with the rest of the program. So my plan is to call a hiatus for the test group for the month of December. I'll go about getting everything finalized and after the first of the year we will have a real beta test. That should not take long. I will also get the Real Time tool ready and we will start a new test as soon as possible, perhaps even concurrently with the other test. When that's done I want to meet with interested imagers and together we will finalize the imaging features for the Pro Edition. The good news is that SkyTools is like a big pyramid and all this stuff is that little part at the top. So I am hopeful we will see a release early next spring.

Crazyness

I've been waiting to have some sort of progress milestone to report, but it's been so long that I felt I had to write something.

How are things going with SkyTools 3? Great! Um... Terribly! I suppose it's the usual combination of the two. The testing end has gone very well, and I am very happy with the program. But this is taking way longer than I had hoped. I mean, it's just plain embarrassing.

I have done this sort of combined development/testing before and it has worked very well. Not only do I fix bugs but I use the feedback from the test group to help me find the rough edges and polish things up. Sometimes I even get inspired and make sweeping changes or add unplanned functionality. This is something that a software engineer under the thumb of management and up against a deadline would not typically be able to do. That's usually a good thing for me, at least from the standpoint of producing the highest quality software possible, which is something that is very important to me. But I'm finding that as SkyTools becomes more and more complex that it is becoming more and more time consuming for me to make sweeping changes. When in the past a major rewrite or unplanned feature might have cost me a few days to a week, this time around it's taking a lot longer.

I have spent the last few weeks redoing the Database Power Search. It started innocently enough: I just needed to work in the new visual detection difficulty filter. That would take a day or two at most. But then I started to realize how much more this tool could and should be doing. Now that dialogs can be larger than 640x480 I had the screen space to make this thing really powerful. Not only that, but I realized that if I split the "Deep Sky" search tab into two tabs (Galactic and Extragalactic) I'd have even more room to add functionality. So first it was this, and then that, and then one more little thing, and all the while the beta test has more or less come to a halt. Like I said, it's embarrassing!

But I keep reminding myself of one thing: a year from now few people will remember the delays. What they will remember is that really cool Database Power Search tool that finally truly lives up to it's name.

That Syncing Feeling

Unfortunately things have bogged down. I could blame it on the kid's school starting, Mrs. Skyhound going back to work, various appointments, etc. But the real problem is that the new Sync feature is, well... out of sync.

The Sync feature allows the user to synchronize their data between two computers. The main point of this is so we can easily move our observing preparations onto our laptop to take into the field, and then easily move the results, such as log entries, back to our desktop. It's a sort of non-destructive backup/restore process, that seamlessly merges the data on one computer into the data on another without losing anything in the process.

Unfortunately, doing that is a huge pain in the ass. Actually, since we are talking about multiple databases here: observing lists, images, logs, notes, telescopes, locations, charts, observers, object notes, web links, etc., it's more like a huge pain in the ass times 50.

I originally did the groundwork for this code last fall. In the meantime, I dramatically underestimated how much of it had been completed back then. Not only that, but changes I have made since have to be folded in and it's been a bit like folding bowling balls into a soufflé. Worse yet, I discovered that I had cheated last fall and much of the code didn't really do what I wanted it to do.

** Sigh **

So while the beta test team is hopefully enjoying some time away from SkyTools I am working hard trying to get, well... back in sync.

The Campground Commute

The Skyhound kids wanted to go camping again before school started, but the Mrs. and I were too busy for a long trip away from home. So we hit on the idea of camping in our own forest here in the Sacramento Mountains. After all, people come from a long way to camp here so why shouldn't we? So we pitched our tent at a local campground less than 0.8 miles away from home! I spent several days commuting to work from the camp site.

I've been hunkered down for the last few days trying to get the next test version ready. The backup and restore functions are finished and they successfully restored my own SkyTools 2 files, which was no small feat. In addition there have been a lot of minor fixes.

The hitch is that I have finally decided to dump the InstallShield installer. Those people are nothing but money grubbers anyhow. Their business model is designed to milk large corporations of significant sums of money. I am not a large cooperation with large sums of money. I wish I had never given them any of mine in the first place; their installer has been nothing but trouble from the start. What really ticks me off is the way they lured me in with what appeared to be a fully functional program, but hidden deep inside were intentional limitations meant to make me pony up more money for a more sophisticated version.

So the next task is to create a new install script using the NSIS installer. But I'm too burned out to start the process... I miss the tent! It kept me from working too much. Time for a break.

Beta Moving Along

We've been testing the standard edition for about a week now. It's been pretty harried. There are sixteen testers and it can be difficult to keep up with them. They have generated a lot of bugs for me to fix! But I don't mind that because fixing bugs is usually pretty simple and often fun. Sort of like doing detective work. You never know where it'll end up. And I can usually fix a bunch of them pretty quickly, which gives a great feeling of satisfaction.

We just did our first test of the new automatic program update, with mixed success. Lots of bugs got fixed, but the update itself was pretty rocky. I have a list of things I want to do better the next time, but of course they won't have the fixes in hand until they finish the update! So we'll have to muddle through at least one more time.

Even with all that going on--loads of emails, and fixing bugs--I've been spending most of my time struggling with trying to finish part of the program that's been giving me trouble for some time. I've gone back and forth on the design, wasting many hours of work, trying to get it right. Today I wrestled the monster to the ground and I think it has finally succumbed! I have some cleanup to do tomorrow and then I'll be free to move on to other things.

I want to get the data backup and restore features finished so we can start testing them, followed by the new sync feature, which will transfer data between SkyTools installations to bring them in sync. This kind of database work can be tedious and I'm not really looking forward to it. But when I get those two things done it will be clear sailing after that. I'll be able to finish up some of the more interesting new features and testing them should be more fun too.

Now Taking Volunteers for the Standard Edition Beta Test

It was a pretty exciting day as we officially started taking volunteers for the standard edition beta test. I will choose the primary team members by Tuesday and put the CD-Rs in the mail on Wednesday. Soon after the fun will begin!

The beta test web page is here:
http://www.skyhound.com/st3/st3beta.html

We've already had quite a few volunteers and I expect many more over the weekend. So far the majority were SkyTools 2 beta test team members and I'm excited to work with them again. It'll be like getting the old gang back together. I don't get to work with other people on SkyTools much; working with the beta test team is something I very much enjoy. That and it will all soon be coming together, which is enormously satisfying. SkyTools 3 has been four years in development (too long by any standard). I've been through a lot during that time and faced quite a bit of adversity, from corrupted data files to lying very near death in the ICU after a botched operation. It will be truly wonderful to finally see this baby out the door.

Standard Edition Beta Test set for 15th

I had a nasty go-round with the Quasar database, but I'll spare you the details. Now that the databases are all but finished I've been working on getting the standard edition of the program together to be put on a CDROM. I have also been working on making sure SkyTools can find all of its files in their new destinations (scattered all over the computer thanks to Mr. Gates) and have been preparing the program for proper use with Vista.

All in all, things are going well! I bet the boss (Mrs. Skyhound) $100 that I could have the beta test process moving along by the 15th. I will be posting a call for testers on or before that date. After all, a hundred dollars is a hundred dollars!

522,066,543

Last week was so annoying that I was thinking of writing a post entitled Open Cluster F**k. I intended to redo the open cluster positions by hand, but decided I'd have one last search of the literature before I took that fairly large project on. As it turns out I found a resource that had everything I needed to greatly improve the cluster data! I must admit, however, to some frustration with myself for not being previously aware of this catalog and with nearly missing it entirely. But there was also a catch (there always seems to be one). In the new catalog each cluster is identified with only one designation and the positions of the clusters had changed, sometimes dramatically. That made it difficult to match them to the clusters in my existing database (this is unfortunately a requirement). So I had to write yet another program to do the matching. It turned into a nasty little problem. Sometimes it feels like matching objects is all I do. I need a break!

At 4 AM one morning I awoke suddenly, realizing that the power was out. I worried about my now-silent computer that had been so busy crunching stars while I slept. Had the UPS shut it down gracefully? I had a look outside and the sky had cleared. Imagine having the lights go out in the surrounding community and you can't tell the difference! I am very lucky. The other amazing thing is that you could still clearly see everything in my observing area, illuminated by nothing but starlight. One thing I discovered when I moved to the top of a 9000ft mountain is that a "dark" transparent sky isn't really dark at all. But I digress.

I waited in vain for the power to return the next day. It didn't come back until 24 hours later, and I was again asleep. The next day I had to go to Las Cruces, so I lost two full days of work. By the time I got back to it, I had trouble remembering what I had been working on so enthusiastically. I hate that.

Anyhow, once again I can report that it all turned out well in the end. The Open Cluster database is improved dramatically with data that was not available when I built ST2. And my computer picked up where it had left off, marching right on with matching those millions of stars for the new stellar database.

Speaking of that, I have to admit to some second guessing regarding my project to create a much larger stellar database. People keep asking me, "Is that really necessary?" This is what always happens when I come up with one of my big ideas. It seems like everyone I know tries to shoot it (and me) down. Don't get me wrong--they mean well, but I have this tendency to go out on a limb with some new innovation and most of the time the people around me don't see it the way I do. Add to that my own fears and insecurities about being out there on that limb and it can be disheartening. Fortunately most of my innovations have turned out for the best, but for a few spectacular exceptions, But the "proof is in the pudding" as they say, and I naturally worry that I'm making a mistake, particularly when it is a long-term project. On the one hand having a much deeper database seems like a no-brainer, but on the other there is that nagging question: do people really need that many stars?

What if I build it and they don't come?

The Pudding Comes out of the Oven
At 5:33 PM today an historic moment occurred (at least for me). The last star was added to the new "pro" stellar database, bringing the total number of stars to over 522 million. Yowza! Leave it to me to be off by a factor of two in my estimate! Apparently I overestimated how many stars would be excluded due to various reasons. SkyTools 2 in comparison, like all other current software, ships with a mere 20 million (or so) stars. So is this a revolution in the making?

Oddly, I also managed to get my estimate for the final size of the database wrong by a factor of two. Thankfully, I overestimated that one. The map database comes to just under 4.3 GB, which is small enough to fit on a single DVD-ROM. This development opens up some interesting possibilities with regard to how we may offer the "pro" version. But again, I digress.

As the database neared completion I was able to play with the parts of the sky that were available. I used ST3 a lot to check out the new open cluster positions and sizes. It looks great. And there sure are lots of stars! There is also a more subtle effect: so many more of the stars now have colors that the display is very pleasing. I got used to it very quickly.

Then I had the "wow" moment. I was looking at NGC 7142, an open cluster dear to my heart, in the ST3 Interactive Atlas when I had the idea to compare it to the old ST2 view. I fired up ST2 and was immediately struck by the difference. As you zoom in with ST2 you quickly reach the point where the existing stars merely get bigger and no new stars appear. You can see for yourself below:

Click here for a much better comparison

After spending so much time with ST3, not only did it feel like most of the sky was missing in ST2, but the position and size of the cluster is, well, rather embarrassing! So do I feel like I've wasted my time with these databases? No Way!

What a relief.

Working Vacation

Well, sort of. This past week Mrs. Skyhound has been away at a conference and the Skyhound kids are out of school. Between feeding them, clothing them, taking them to their baseball games, following both the Stanley Cup and Louis Vuitton Cup finals with them, and separating the two of them when they fight, I haven't had a lot of time to work.

Nonetheless there have been some important milestones. In addition to working in here and there the tying up of little loose ends, I added even more improved data to the planetary nebula database (it's really looking good).

But the big news is that I finished the end-to-end testing of the new stellar database and it passed with flying colors. The final count was 16,330,450 stars, including over 50,000 variables and 72,089 multiple star systems. I don't have a final count for the total number of double-star pairs, but it is likely close to 100,000. I also ran many tests on special cases, such as the long-period binary pair BL Cet and UV Cet, Barnard's star, etc. Everything is go.

It was time to start creating the full database for the "pro" version, which will add large numbers of faint stars down to magnitude 20. There are a total of 24 declination bands, of which 3 are now complete. At the current rate it looks like it may take a week running continuously to finish the database, bringing the total number of stars to something around 250 million. SkyTools is really impressive when targeted in the bands already completed. It seems like you can zoom in forever and the stars just keep coming!

The Story of Skyhound

When I started my fledgling software business back in 1990 I needed a name. I loved the name Sirius Software (a pun would be absolutely perfect for my sense of humor) but it was already being used. I wanted something unique that had "soft" in it for "software." So I started going through constellations and stars adding "soft" to them, such as LyraSoft, AltairSoft, etc., until I came across one I liked: CapellaSoft. I didn't realize at the time that for many around the world I'd in fact named my endeavor GoatSoft!

At that time the commercial Internet was just getting started and I wasn't ready as yet to have my own domain name. When the time finally came I checked for capellasoft.com and discovered to my great dismay that it was already registered. Some guy in Phoenix had it, although he wasn't using it. No doubt he'd registered it with the idea that I'd one day buy it from him. That really ticked me off; I hated the idea of being extorted for my own name. About that time we were thinking about moving from California to New Mexico, and we made a car trip across the southwest. On the long drive I was trying to figure out what I wanted to do with my new web site. I knew I wanted to split it into two parts, one for the business and one for observing. But try as I might I couldn't come up with a good name for the URL. With capellasoft.com taken I was looking for something more unique and interesting.

As we drove across southern New Mexico we passed Rock Hound State Park. I thought, if there were amateur rock hounds, why couldn't there be amateur sky hounds? When I got home I checked the domain name: skyhound.com was available. I also did a search on skyhound using the search engines of the day, and I got only two hits, both from Italy. Apparently there was a sci-fi comic book that had a space shuttle in it named Skyhound. But that was it. I knew I'd found my name. Eventually it was too confusing to have CapellaSoft at skyhound.com, so I went ahead and changed the name of my business to match.

After I registered skyhound.com I realized that if I was going to coin a new word then I had the chance to define it as well. So I added this at the top of my new web page: "Skyhound -- noun: someone with a keen interest in all aspects of the sky--from visual observation to scientific curiosity."

This morning I typed Skyhound into Google. There were 13,800 hits! It's not all me though--there is a wireless internet company called Skyhound, which judging by their logo is a takeoff of skyhoundz competition, which is about dogs catching frisbees.

I used to be able to sign up for web services and email addresses using the nickname skyhound, but now more often than not it's already taken. Although on the one hand that's a bit annoying, on the other I have to chuckle and shake my head. I wonder how much in total the guy in Phoenix paid year after year for capellasoft.com?

Progress and Frustration

It's been a couple of weeks since my last post. There's been quite a lot going on. My family has kept me very busy as the school year winds down and I've had a nasty cold. So far my progress report for this month is a mixed bag: things have been going slower than I'd like, yet I have undeniably moved forward toward that ever brightening light at the end of the tunnel.

For a while there it seemed like things were moving at a snail's pace and I was pretty frustrated. I bitterly recalled claiming here that making the chart cursor ignore objects when the shift key is depressed would take "two minutes." So I decided to time myself. Sure enough, it only took 1:45 from opening the editor to testing it successfully. But before I drew too much satisfaction, I realized that I needed to add the same code to two other charts, meaning it would in fact take three times longer than I had estimated!

Sigh.

I turned most my attention to the observing lists, adding a bunch of new columns. It seems like not a day goes by that I don't add a new one. In fact, I thought of a new one last night that I need to add today. This is one of those annoying things that appear as one item on my ToDo list, yet like a cosmic ray explodes into a shower of little items to be checked off. The most time consuming part isn't changing the code, it's coming up with nice-looking icons for the columns that require them. For instance, I spent a lot of time on the observing priority and observing status icons. For the status I ended up with a little Keck dome open to the night sky for objects that need observation, and a daytime shot of the dome closed for objects that have been observed. They look nice but I'm not sure I like them.

The priority icons are just awful. I hate them! So I'll have to try another approach, maybe simply going with the numbers 1, 2, and 3. I dislike wasting time on such trivialities, but in the end it's often the little details that count the most and the problem with being near the end is that you can't put them off any longer. Fortunately I like how the 5-star ratings came out.

One thing that's satisfying is making a little leap of insight. I had a customer email me some time ago with regard to the needs of imagers, and I read what he wrote but didn't really get what he meant through my sometimes rather thick skull. Fortunately while I was looking into imaging from the "what can I compute?" point of view I had a tiny epiphany: it's great to know what your chances of detecting an object tonight are, but it's even better to compare that to the best circumstances. And that applies to visual observing too. In other words, if tonight detecting M98 is "challenging" it's immensely useful to know that on another night it may in fact be "easy." So I added a new column that tells you how easy the object is to detect on a the best night for comparison. Unfortunately it takes up quite a bit of screen real estate. So I'm adding a new column with little colored-circle icons that summarizes the comparison: green means that tonight the object is at prime visibility, yellow means it is degraded somewhat, and red means wait for another night. And yes, this is pretty much what had been suggested (smacks forehead with palm).

I've also been working on the deep sky databases. It turns out that the LMC globular clusters were missing. Being an ignorant northerner, I didn't even realize there were LMC globulars! The source catalog I used was for Milky Way clusters (duh). Some are quite bright and nice, for those far enough south to observe them. I've also added the MASH planetary nebulae and updated positions and magnitudes for other planetaries. I'm currently working on the open cluster database, primarily improving positions. Fortunately the databases are going fairly quickly.

That's NGC 2210, a "new" LMC globular cluster above.

Anyhow, like I said it's a mixed bag to report: I've had to let go of the idea of starting the beta test at the beginning of June, but on the other hand things are moving along without any more major setbacks. I'm not going to try to predict anymore when It'll be ready to start testing other than to say it will be this summer for sure.

The Problem with Loose Ends

The danger with loose ends is that sometimes you can pull one and things unravel.

So there I was early last week, happily finishing things and marking them off my list when the next thing you know I'm off on one of my parenthetical random walks. I was adding new columns to the observing lists portion of the main planning tool: one for status (observed, not yet observed, re-observe) one for setting observing priorities, and a third for 5-star object ratings. With the ratings column came a small problem. The ratings (and notes) that are displayed depend on the selection of a note group, and I hadn't placed a group selection on the dialog yet.

I dislike designing the layout of dialogs, particularly when space is at a premium, and I really hate having to do the same one more than once. So naturally I started thinking ahead to what other items might need to be worked in. This lead me to thinking about the new imaging features. I knew I'd need a toggle of some kind to switch between visual and imaging modes. What else would I need? It was an innocent thought: perhaps I should look into imaging a little bit more so I'd have a better idea what I wanted to accomplish on this dialog.

Before I knew it I was knee deep in readout noise, exposure times, and signal to noise ratios. My desk was covered with pages of algebra. And even my dreams had CCDs in them.

The point of all this was to find imaging analogs to the optimum times and detection difficulty that SkyTools uses as the basis for planning visual observation.

Visual observation is all about contrast. For the best contrast the observer requires the darkest sky possible. So the important questions for the visual observer revolve around how dark the sky is.

Imaging is all about signal to noise ratio. Imagers can cheat a bright sky with a longer exposure or by stacking many exposures. Many of the important questions for the imager still depend on how dark the sky is, but they are more subtle. The visual observer may ask, "Should I observe Sh 2-1 tonight, or wait for a better night or darker location?" while the imager asks, "If I observe Sh 2-1 tonight, how much more time and effort would it take compared to a darker night or location?" Therefore my job to support imaging is to quantify "time and effort" in the observing list. Optimum times also mean less: more important is the observing window and its duration.

I also realized along the way that my new ability to model the sky brightness can be folded into a useful tool for calculating exposure times.

Anyhow, it's back to loose ends again. But I'm really happy about my diversion into imaging this past week: the new imaging features were very hazy and unclear, and now they are beginning to come more sharply into focus. I can't wait to make it become a reality!

Loose Ends

Now that the primary pieces of SkyTools 3 are in place, the next phase is to tie up all the loose ends. These loose ends range from projects I got sick of working on and never finished to small suggestions from users that I promised to incorporate. This is a huge milestone for me because I no longer have to face down large daunting tasks with major problems that must be solved with little idea of how long they will take to finish. From here on out it's all downhill, and a lot more fun.

I spent some time this morning making a giant todo list. I poured over my notes, including in the list unfinished projects, minor ideas I have yet to implement, and user suggestions I committed to incorporating. The final list has 71 items. That may seem a little daunting, but it isn't really. There are only four or five items that amount to more than a days work. The vast majority can be accomplished in lest than an hour. Some will only take a few minutes. Of course, history says at that at least one will unexpectedly take a week... (or two)

The best part of all this is psychological. It can sometimes be difficult to get up to speed on a big project, and when it gets mired down in unsolved problems, already a week behind schedule, it can be hard to keep up the momentum. But a todo list is great! Even on the most beautiful spring day when all I can think about is sitting in the sun reading a good book, I can easily get myself up for doing the first little thing on the list. And the next. And the next. At the end of the day it feels great to know that I finished 12 items!

Probably the biggest remaining item is my new Observing List Wizard. Ok, I call it that, but technically it isn't a Wizard, so I really should stop. But it seems so boring to call it the Observing List Generator, or some such. How about the Observing List Suggester? Yuck. Sometimes I wish I had someone to help who's only job was to come up with great names for things.

Anyhow, I wanted to have a more simple method of creating observing lists than the Database Power Search. Now that SkyTools can take a telescope, location, and observer and rate how difficult it is to detect an object, I thought it would be great to have a tool that could generate a list of objects for beginners, or a list of observing challenges, etc. The primary idea being a very simple and general set of inputs which would create a randomized list of 'n' objects that meet the criteria.

Another one will be finishing the preview pane I added to the chart preferences dialog. The idea here is to give you a preview of how the element whose color or font or line style you are changing will look on the chart.

Then there's adding the new detectability criterion (obvious, easy, ... challenging) to the filters on the events and ephemerides tabs.

The planetary nebula database should be updated with better magnitudes and some new data.

The open clusters need to have more accurate positions. I'll first get them from SIMBAD then redo any that are still off by hand. Good thing there aren't millions of them!

Then there are little things. Like my new Thumbnail Viewer. This viewer displays more than one chart arranged as thumbnails. You can customize it as you want and it will display either Atlas or eyepiece charts. I finished the thing 6 months ago, but never got around to wiring it into the program.. At the very least it should be great for displaying the objects in an observing list. And maybe even for lists of events. Regardless, all I need to do is place a line of code at the right spots.

There are other little things like making the cursor ignore objects on the charts when the shift key is depressed. I mean, how long could that take? Two minutes?

Once I get most of these things done I'll bring it all together and see if I can get it on a CD (or two DVDs for the imaging version) and I'll start putting together a beta test group. I really look forward to working with the volunteers!

The Context Viewer is Finished

The Context Viewer is finally done. This is a pop-up window that is attached to the SkyTools Interactive Atlas. It displays a simulated view of the sky as seen in a telescope.

The new SkyTools 3 telescope simulations employ a more sophisticated model to determine the visibility of objects at the eyepiece. A more rigorous approach is used that includes a realistic sky brightness model and for diffuse objects a contrast algorithm now estimates visibility. The sky brightness can be drawn in the view background, so for instance during the day not only are most stars missing, but the background is blue.

At first, when I'd finished creating the viewer window, I was a bit disappointed. At that point all it did was draw a telescope view of the same target as the Atlas. It was only today when it came to life that I began to see it differently. You know, I come up with an idea like this and it only exists in my imagination. Then one day, after much work and tedium, it's done; it actually exists! It's enormously satisfying. But better yet, I'm always the first person to get to play with the new toy. Sure, there are always rough spots, and sometimes things don't live up to expectations, but playing with a new tool always opens new doors, and new possibilities. It's hard not to be excited about it. I've only had an hour with the Context Viewer so far, but I already like it a lot.

Here's how it works: there are two buttons on the context viewer tool bar. One locks the viewer to the atlas. In this mode it follows the atlas around the sky, always targeting the same object or position. The second button locks the viewer to a telescope, if connected. This is where things start to get really cool. Wherever the scope is pointing, that's what appears in the view. And for GOTO scopes it works both ways--if you move the viewer the telescope follows. An eyepiece circle appears on the atlas representing the current position and field of view displayed in the viewer.

Perhaps the best part is that using the mouse you can grab the eyepiece circle and drag it around the atlas. Imagine working your way through the Virgo Cluster. The atlas is configured so that the entire cluster is displayed. As you drag the eyepiece circle to each galaxy, the context viewer displays a simulated view at the eyepiece. And better yet--if there is a telescope connected as you drag the circle to each galaxy your telescope will follow. A quick check of the context viewer to see which galaxies should be visible, then it's off to look through the scope. And for small difficult objects, being able to quickly identify the stars in the field should make finding them at the eyepiece a snap.

Anyhow, that's the basic idea. It will likely mature as I play with it and we do the beta testing. I'll also be adding a similar capability for imaging, including support for off-axis guiders.

I'm a Father to be of 250 Million!

Well, it's been a while since my last post. The kids had their spring break last week so I took some time off, but mostly I've been hunkered down in the trenches.

Detailing my slog with the stellar database might be about as interesting as reading about someone watching paint peel off the wall. But since this is what I've spent the majority of the last three years working on I should at least say something about it.

I didn't set out to redo the stellar database from scratch. I had a lot to build on as a starting point with the SkyTools 2 (ST2) database. But I get suckered into these things as I go along, slowly pulled ever deeper into the pit by the catalog monster. My primary goal was to include the full Washington Double Star (WDS) catalog. Simple enough. ST2 has many of the same pairs, but they are derived from the CCDM catalog. This has meant that the commonly used WDS identifiers haven't been present in the database, and that some pairs were simply missing. The thing is, the WDS has always been a huge mess! That's why I avoided it in the first place. Although much improved in recent years it is still rife with errors and mistakes. In addition to incorporating the WDS I needed to update to the latest General Catalog of Variable Stars (GCVS) and its accompanying New Suspected Variable (NSV) catalog.

I was also excited about the prospect of incorporating the USNO-A2.0 catalog to extend the database much fainter. I asked for and received permission to do this from Dave Monet.

But what really set things off was the publishing of the second version of the UCAC catalog. This catalog goes almost as faint as the old Guide Star Catalog (GSC) but with much improved data, including accurate proper motions. Throw in the new USNO-B1 catalog with stars and proper motions down very faint, and you have a lot of new data available.

So I set about incorporating all these catalogs into the ST2 database, in one way or another. The basic idea is to compile all the available data into the SkyTools database. That may seem straight forward, but I go about it in a very different way from other developers. Most astronomical software acts like a data engine, overlaying the data from various sources. But what I do is incorporate ahead of time all the available data into a custom database, and this includes double star components.

Take a WDS pair for instance. In fact, they may be two pairs that share the same primary star. One pair is close and the other distant. So we have the primary (A) star, with star B 2 arc seconds away at some position angle measured in 1902. In addition we have star C at a distance of 16 arc seconds and a position angle measured in 1991. The GCVS claims that the B component is variable. But the magnitude range of the variable star doesn't match the magnitude listed for the B component. So what's wrong? With the advent of these new stellar catalogs even the fainter stars have proper motions. The proper motions allow me to run the stars back in time to see how their relative positions have changed. I do this to find which USNO-B1 or UCAC2 star macthes each of the component stars. Then I run the stars back in time to 1991, and even 1902. And voila! Component C moves in close to the primary and star B moves farther away. The variable star referred to by the GCVS is actually star C, not star B. Only back in 1902 they were switched in position!

When you consider how many pairs there are in the WDS you might begin to understand the size of this detective work. Mostly I created programs that match the stars from various catalogs and solve these mysteries, but in many cases I needed to do the detective work by hand. So I created an interactive program and spent many hours matching stars, making thousands of my own corrections.

This sort of thing goes on and on as I match stars from all the catalogs, including HIPPARCOS, Tycho-2, GSC, UCAC2, USNO-A2.0, USNO-B1 and all the cross reference catalogs like the SAO, BD, and PPM.

I'm now at the point where it is all finally coming together in a final form. The stars with more than minimal data appear in my primary reference database. I fold the less interesting and/or fainter stars into the mix when the mapping databases are created. There are actually three of these databases, one that goes down to 7th magnitude, a second that goes to 10th, and a third that in the past went to 15+. With the addition of faint USNO-A2.0 stars, this latter database can now be optionally extended to magnitude 20.

So what does this all mean for the user? It means more double stars, more variable stars, more accurate positions, many more stars with colors and proper motions, and the possibility of a much larger stellar database to support imaging.

To take advantage of all this I'm adding two new tools to the charts. One will create rudimentary color-magnitude diagrams for stars within an area. The other will plot the proper motions of stars with small arrows. These tools should provide advanced users with the ability to judge for themselves whether or not a grouping of stars is a real cluster, or to investigate how the stars in a cluster move together over time.

But the most obvious change will be the size of the database. SkyTools 3 will be made available in two versions: "standard", and "professional" (the names may change later, but you get the idea). The new imaging features will be Incorporated into the pro version, along with an extended database. The pro version will ship on 2 DVDs. The number of stars available will jump from the current 18 million, to as many as 250 million!

Except in a few selected areas of the sky I've been using the standard database for testing because I can regenerate the 15th magnitude mapping databases in just a few minutes. I estimate that it will take 5 days running continuously to generate the final "pro" database! So I'm waiting until I have the stars debugged before I make the final run. But that time is coming; the SkyTools 3 pro database will very soon be born, and after all this work, I'm pretty excited about it!

A Trail of Two Labels

I took some time off over the weekend for a family hike on one of the new trails opened up by New Mexico Rails to trails. It was relaxing.

So I have a confession to make: I've never been happy with the coordinate grid labels. I've recently been working on improving the grid lines and adding new meridian and ecliptic lines. And right there, just below where I was working, loomed that pesky grid label code. I couldn't help myself, I just had to see if I could finally make it better.

Digging in this code brought back memories. I know I made numerous improvements for SkyTools 2, but what came flooding back was when I originally wrote it. I clearly recall a long walk with my wife, a friend, and her dogs. It was a pleasant late-spring afternoon at San Diego's Lake Murray, not far from where SkyTools was born. While my wife and friend talked I spent the whole time trying to wrap my head around how I wanted to tackle coordinate grids and their labels. As the Sun set and the sky darkened, Comet Hale-Bopp was visible in the clear western sky.

As much as that is a pleasant memory, I hate this code! It does not draw labels that look as nice as I want. I considered for a bit, trying to figure out how to make it better. I wasn't sure what to do, but one thing seemed clear: rotating the labels in concert with the grid would be a big step in the right direction.

Labels rotated to arbitrary angles... Ever notice SkyTools doesn't do that? There are a couple of reasons, but primarily this is due to the use of my "smart label" module, which keeps labels from overwriting each other. This module is built on the idea that the labels are defined by simple rectangles aligned with the screen. But I'd had enough of that. I didn't know how, but I figured there had to be a way to add rotated rectangles to the smart label code. So I created a new kind of label that could be rotated to any angle and simply had faith that this smart label problem could be solved.

Still stuck on how I wanted to do the grid labels, I turned to the other labels that cried out for arbitrary rotation: the object trails. Before you know it I had nice little trail labels, all lined up. Well, actually, they didn't line up that well. There was some variation in the angle for each label, which made them look rather messy. I tried a couple of things, and then I realized that I should simply force all the labels to have the same angle until they rotated more than, say, 10% from each other. And there you had it: nice neat labels all lined up!

But what about making them "smart" so they won't overlap? It took me the better part of a day, and I had to try three different approaches, but I finally came up with a hack (umm... I mean clever algorithm) that made it work. Whew!

This all may sound trivial, but it is the little things that in the end make all the difference. Oh, and the grid labels came out nice too. Rotating the label to match the grid makes it much more clear which label goes where. That and a little TLC has them looking much better!

I'm off to the stellar database again for another go-round. I need to finish up the updating of the cross reference databases. And it turns out there are stars showing up in Scutum that aren't really there...

The Random Walk

Like a photon in the Sun, sometimes my programming follows a random path toward completion. My work becomes parenthetical, where I start one thing, which leads to a change to something else, which then leads to some inspiration to fix it for the better, which reminds me of something I always wanted to do, which in turn leads me further away from the task I began, and so on. My great fear is that I'll get caught up so far away from where I started that I won't be able to find my way back, or even if I do, I'll have lost the inspiration that was driving the original project. Yet somehow I always find my way back and I always rediscover the inspiration, although is some cases it was years later! But most importantly, I have learned to trust these detours, because I have never followed one that I would later regret; in every instance they have made SkyTools better.

Recently I started on the last major piece of SkyTools 3 that had yet to be written. It's what I call a Context Viewer. This is a chart window that is attached to the Interactive Atlas, as a sort of pop up. This window will display an eyepiece view for the selected telescope and eyepiece combination. An indicator appears on the Atlas that you can drag around to position the eyepiece view. It will also support cameras. Building this window might seem like a big project, but in fact I can borrow code from elsewhere (primarily the new Thumbnail viewer--but I'll save that one for another time). It's really just a matter of copying and pasting code and then making some modifications, something I find simple, yet tedious.

One morning I got to work and simply couldn't bring myself to continue on it. This happens from time to time, and I know I'll spend the rest of the day staring out the Window or posting to Usenet if I don't give in and find something more appealing to do.

I'd been copying a code snippet from the Chart Controls dialog when I had one of those crazy ideas. Could we just get rid of that dialog entirely? I had a look: sure enough, the only controls that were not already on the charts were the coordinate system and map projection used by the Interactive Atlas. I realized they could be moved to the tool bar. The scenario selection is also stored there, and I briefly considered removing it, but decided to move it to the tool bar as well.

Moving these little-used controls to the tool bar was one of those small changes that was at the same time rather dramatic, and I was pleased by the result. Now more prominent, I think they will get much more use. It seemed like this would also be a good opportunity to add the galactic coordinate system as planned. Of course, once I had the selection labeled "Galactic Coordinates" in the Combo Box, I needed to actually add this coordinate transform to the mapping code! While doing that I realized that I should also have galactic coordinates as a grid selection. While adding them to the grid I remembered a question someone asked on a forum about software that could display what was below the horizon. I had the sudden realization that I should split the horizon coordinate mode into two modes; one that displays simple horizon coordinates (and extends below the horizon) and one that simulates the sky with extinction, refraction, etc. So I did that. But I also needed to make changes in the horizon line... and of course I promised to add a meridian and ecliptic, and I should probably plot more "cardinal points" such as the Galactic Poles and Galactic center.

Somewhere in there I discovered that in the simulation horizon mode my new shaded sky no longer matched up with the horizon as drawn. So off I went to my new sky shading module. It took a full day to straighten that out!

And what was I supposed to be working on again?

Next time: how all this led to a breakthrough in object trail labels!