SkyTools 3 is done. Both editions were completed last week.
It's a bizarre experience. There I am working toward something that has taken four or five years to accomplish, depending on how you count it. As the final months come the days-off get fewer and fewer. As the final weeks come the days-off disappear completely and the work days grow longer. Constant coding turns into creating artwork for the packaging. The last few days become a blur. Code that was prepared months ago that is only used in the final product is at last integrated. The bugs continue to flow in from the test team and must be squashed immediately, even if it means a late night. Plans that were made years ago are put into motion. The website must be updated. Difficult but critical questions have to be answered, such as, "How many of each edition should I have made?" And above all there is that constant, ever present question: "Did I forget something?"
It all builds to that that final day when the final CD master is burned, somehow always only ten minutes before the UPS overnight pickup deadline. The package is sealed and raced to the drop. And then its over.
It will be two weeks of waiting before the packages are returned and we can start shipping.
People keep asking me, "Is it done yet?" and I just stare at them, dumbstruck. Saturday morning comes and I feel guilty for not getting up early to go to work. My desk is a mess. I guess I should clean it off now. The "to do" lists that ran pages every day now only run a page for the week. It is peaceful. Serene. But that constant, ever present question remains: "Did I forget something?"
It's like a Shuttle launch but sort of backward. There is a slow ramp up of thrust that builds to the great deafening explosion of launch, followed immediately by an equally deafening quiet and a feeling of weightlessness... of drifting.
And of course, "Did I forget something?"
Sunday, December 7, 2008
Wednesday, August 20, 2008
Asteroid Excess
We're about ready to start the final testing of the "Pro" version of SkyTools. There are just a few loose ends left to tie up, such as the new serial number code, enhancing the installer, etc.
Oh, and all those pesky asteroids.
When I first wrote the asteroid code for SkyTools there were between 8000 and 9000 known. By the time SkyTools 2 came out there were so many that I only put the first 10,000 numbered asteroids on the CD because there wasn't enough room for them all. Today there are nearly 200,000! That's so many that drawing asteroids on a chart is really bogging down.
Here's how the asteroids that are shipped with SkyTools work: asteroids use osculating elements, which are a sort of cheat. These elements would describe the orbit of the asteroid perfectly if there were no other bodies in the solar system. But with all those other asteroids and planets out there the reality is that each asteroid gets tugged around over time, changing its orbit. So the orbits we use are sort of freshness dated. They are good for about 40 days. What I do is run whats called an "n-body" program that carefully computes the position of each asteroid over many years, including all the tugging. Every 40 days a new set of orbital elements are generated. When you ask for the position of an asteroid SkyTools finds the set of elements closest to your time and uses it to compute the position.
The supplemental asteroids, which are downloaded over the web, work in much the same way. Each new set of elements you download gets added to the database. Over time you will build up a database that covers months and even years.
This all works pretty well, except for how long it can take to run through the nearly 200,000 asteroids sorting out the ones bright enough and within the bounds of your chart. Fortunately I had an idea come to me in the middle of the night a few months ago. It's one of those ideas that I would really like to claim is perfectly brilliant, yet it's so obvious I probably really ought to be ashamed I hadn't thought of it before.
The idea is to store a set of positions and magnitudes along with the orbital elements for each asteroid. Since the elements are only used for a known 40-day period we can compute a list of positions over that period (using the elements). When we want the position we interpolate to get a good idea of the position and magnitude of the asteroid. Only if the asteroid is on the chart do we go ahead and compute the accurate position. Interpolation is cheap so the end result is that the asteroids plot a lot faster. How much faster, I'm not quite sure yet. But I expect it to speed things up a lot.
Unfortunately there is a downside. The positions stored with the elements have to be pre-computed. I just tried downloading and pre-computing all the known asteroids and it took a whopping 8 minutes on my aging computer. In fact, that's why I'm writing this. I'm running another test and it's a long wait...
Fortunately I have some ideas for speeding it up!
P.S. I tested the new asteroid code and it is 12x faster!
Oh, and all those pesky asteroids.
When I first wrote the asteroid code for SkyTools there were between 8000 and 9000 known. By the time SkyTools 2 came out there were so many that I only put the first 10,000 numbered asteroids on the CD because there wasn't enough room for them all. Today there are nearly 200,000! That's so many that drawing asteroids on a chart is really bogging down.
Here's how the asteroids that are shipped with SkyTools work: asteroids use osculating elements, which are a sort of cheat. These elements would describe the orbit of the asteroid perfectly if there were no other bodies in the solar system. But with all those other asteroids and planets out there the reality is that each asteroid gets tugged around over time, changing its orbit. So the orbits we use are sort of freshness dated. They are good for about 40 days. What I do is run whats called an "n-body" program that carefully computes the position of each asteroid over many years, including all the tugging. Every 40 days a new set of orbital elements are generated. When you ask for the position of an asteroid SkyTools finds the set of elements closest to your time and uses it to compute the position.
The supplemental asteroids, which are downloaded over the web, work in much the same way. Each new set of elements you download gets added to the database. Over time you will build up a database that covers months and even years.
This all works pretty well, except for how long it can take to run through the nearly 200,000 asteroids sorting out the ones bright enough and within the bounds of your chart. Fortunately I had an idea come to me in the middle of the night a few months ago. It's one of those ideas that I would really like to claim is perfectly brilliant, yet it's so obvious I probably really ought to be ashamed I hadn't thought of it before.
The idea is to store a set of positions and magnitudes along with the orbital elements for each asteroid. Since the elements are only used for a known 40-day period we can compute a list of positions over that period (using the elements). When we want the position we interpolate to get a good idea of the position and magnitude of the asteroid. Only if the asteroid is on the chart do we go ahead and compute the accurate position. Interpolation is cheap so the end result is that the asteroids plot a lot faster. How much faster, I'm not quite sure yet. But I expect it to speed things up a lot.
Unfortunately there is a downside. The positions stored with the elements have to be pre-computed. I just tried downloading and pre-computing all the known asteroids and it took a whopping 8 minutes on my aging computer. In fact, that's why I'm writing this. I'm running another test and it's a long wait...
Fortunately I have some ideas for speeding it up!
P.S. I tested the new asteroid code and it is 12x faster!
Friday, June 6, 2008
Imagining Imaging
I have some great news. But first some background.
Years ago I had a simple idea: extend what SkyTools does for the planning of visual observations to planning imaging sessions. It seemed trivial because the two kinds of planning are so similar. After all, it's still about observing an object high in a dark sky. It seemed that all it would take would be a few tweaks to make the planner more imaging friendly.
In designing the planner for visual observing my approach was to come up with the basic questions that an observer might want answers to, such as, "Can I see this object in my telescope from my backyard tonight?" Or, "Which of these objects in my list should I observe tonight?" And, "When is the best time to observe this object tonight?"
To answer these questions I borrowed, enhanced, and even developed from scratch science-based models for atmospheric extinction, the brightness of the sky, and the visibility of objects in the telescope. In SkyTools 3 these models are now very sophisticated. To make these models work you must teach SkyTools about your telescope, your observing location, and a even a little about yourself. Many people don't realize it, but the ability of SkyTools to compute the optimum time to observe an object is the critical component. I leverage this ability in every way I can think of to create the various visual observing tools. This optimum time computation, along with the quality of observation assessment that makes it work, are the engines that makes SkyTools go.
For imaging support I turned to the very same process: create a list of questions to answer, develop a science-based imaging model, and allow the user to tell SkyTools about their imaging system. I did most of the model last summer and I was pleased with the results. But a nagging question remained: what was the analogue to the optimum times for visual observing that would serve as the engine for planning imaging sessions? And not only that, but in the end would I be able to offer something more than what is already available? Is there a useful niche for SkyTools to fill in the imaging world? These questions have nagged me for over a year now. I knew that at the heart of deep sky imaging is signal to noise ratio (SNR), but how to leverage it? Visual observing is about a brief moment in time, but imaging is about an exposure that can last many minutes or even hours. And unlike visual observing the brightness of the sky is not always critical. So the questions become, "Should I bother to image this object from my light polluted back yard or wait until I can get to a dark site?" Or, "Is it worth imaging when the moon is up?" These questions simplify to, "If I do image in a bright sky, how many more exposures will I need to get the same result?" And, "Is there enough time to make it worthwhile?"
To be honest with you, even as recently as a few weeks ago I didn't know how to best answers these questions. I had most of the pieces in place for planning imaging sessions, but it had yet to come together. Programming for me is a lot like digging a tunnel through a mountain, where you start at each end and hope to meet in the middle. I've been told that a "good" software developer (or team) sits down and designs the entire project in advance. But I have found, at least for me, that in practice this is usually only a good start. Not only do problems emerge that you had not anticipated, but the very act of writing the code often brings new insights. So as I build the code both from the bottom up and from the top down I try to remain as flexible as possible. The process that unfolds feels like a journey. I'm never really certain how it's all going to work out. Where will we meet in the middle? Will the result be useful? In many ways I feel like I'm just along for the ride, following along where the code takes me. Thankfully this creative process has seldom failed me. (Unlike my predictions for how long things will take!)
I am pleased to announce that my ride is almost done. Things have come together. Important questions now have answers, and I look over what I have created and I am excited by the result. The feeling of satisfaction derived from taking an idea from a mere formless seed to fruition is wonderful. This is what it's all about for me!
We should begin beta testing the Pro version (with imaging support) in a few weeks. I am very excited about sharing it with everyone and equally excited about working with the team to put a nice polish on it.
Years ago I had a simple idea: extend what SkyTools does for the planning of visual observations to planning imaging sessions. It seemed trivial because the two kinds of planning are so similar. After all, it's still about observing an object high in a dark sky. It seemed that all it would take would be a few tweaks to make the planner more imaging friendly.
In designing the planner for visual observing my approach was to come up with the basic questions that an observer might want answers to, such as, "Can I see this object in my telescope from my backyard tonight?" Or, "Which of these objects in my list should I observe tonight?" And, "When is the best time to observe this object tonight?"
To answer these questions I borrowed, enhanced, and even developed from scratch science-based models for atmospheric extinction, the brightness of the sky, and the visibility of objects in the telescope. In SkyTools 3 these models are now very sophisticated. To make these models work you must teach SkyTools about your telescope, your observing location, and a even a little about yourself. Many people don't realize it, but the ability of SkyTools to compute the optimum time to observe an object is the critical component. I leverage this ability in every way I can think of to create the various visual observing tools. This optimum time computation, along with the quality of observation assessment that makes it work, are the engines that makes SkyTools go.
For imaging support I turned to the very same process: create a list of questions to answer, develop a science-based imaging model, and allow the user to tell SkyTools about their imaging system. I did most of the model last summer and I was pleased with the results. But a nagging question remained: what was the analogue to the optimum times for visual observing that would serve as the engine for planning imaging sessions? And not only that, but in the end would I be able to offer something more than what is already available? Is there a useful niche for SkyTools to fill in the imaging world? These questions have nagged me for over a year now. I knew that at the heart of deep sky imaging is signal to noise ratio (SNR), but how to leverage it? Visual observing is about a brief moment in time, but imaging is about an exposure that can last many minutes or even hours. And unlike visual observing the brightness of the sky is not always critical. So the questions become, "Should I bother to image this object from my light polluted back yard or wait until I can get to a dark site?" Or, "Is it worth imaging when the moon is up?" These questions simplify to, "If I do image in a bright sky, how many more exposures will I need to get the same result?" And, "Is there enough time to make it worthwhile?"
To be honest with you, even as recently as a few weeks ago I didn't know how to best answers these questions. I had most of the pieces in place for planning imaging sessions, but it had yet to come together. Programming for me is a lot like digging a tunnel through a mountain, where you start at each end and hope to meet in the middle. I've been told that a "good" software developer (or team) sits down and designs the entire project in advance. But I have found, at least for me, that in practice this is usually only a good start. Not only do problems emerge that you had not anticipated, but the very act of writing the code often brings new insights. So as I build the code both from the bottom up and from the top down I try to remain as flexible as possible. The process that unfolds feels like a journey. I'm never really certain how it's all going to work out. Where will we meet in the middle? Will the result be useful? In many ways I feel like I'm just along for the ride, following along where the code takes me. Thankfully this creative process has seldom failed me. (Unlike my predictions for how long things will take!)
I am pleased to announce that my ride is almost done. Things have come together. Important questions now have answers, and I look over what I have created and I am excited by the result. The feeling of satisfaction derived from taking an idea from a mere formless seed to fruition is wonderful. This is what it's all about for me!
We should begin beta testing the Pro version (with imaging support) in a few weeks. I am very excited about sharing it with everyone and equally excited about working with the team to put a nice polish on it.
Monday, April 14, 2008
Real (long) Time
It's Monday, I have a headache and I don't feel much like working, so I thought I'd finally post something on my blog. Usually what happens is if I think about writing something I decide that working on the program is a better use of my time. That's why you haven't heard from me much in the last few months.
Before I get to the fun stuff, let me say that "it" has happened again. At every point in the process of developing SkyTools 3 I have run into the very same problem, and even I, head deeply buried in the sand, am beginning to see a trend. I have been operating under a very basic misconception. I know this may sound ludicrous to some of you but I naively thought that SkyTools 3 would be easier than SkyTools 2. My reasoning went something like this: ignoring the database updates, which I know is always an enormous job, most of the actual programming was already in place. To add a new feature or extend functionality would be simple. But that is not the case. Instead, as the program becomes ever larger and more complex, just about anything I do to it becomes more difficult, requiring ever more time and testing. It has become rather clear that the methods I had used up to now, which had served me well, are no longer viable. I have begun to rethink everything with one goal: to release an upgrade every two years. Wish me luck.
Looking at the calendar I can see it is already April and I haven't got the Pro version testing up and running yet. The main reason is that, once again, I have greatly underestimated how much there was to be done. Not to mention all the distractions, like spending several hectic days and late nights last week moving my web site to a more reliable host. And even the testing is proceeding much more slowly than our original Real Time testing did. For instance, I released a bunch of new Real Time features last Friday with the expectation that at least some testers would try them out over the weekend. But so far I have heard nothing. I don't know if it's weather, if the testers have lost interest, or what. Regardless, it is a bit disheartening. I'll have to move on. And when bug reports finally do come in I'll have to go back and figure out what I did, where the code is, remind myself how it all works, and in the process leave what I have started in the meantime half finished. Then I'll have a similar problem when I go back to what I was working on.
But as always, there is a pretty big pot for at the end of this rainbow for SkyTools users. My knack for overreaching means that there are many new features, several of which were unplanned. Here are some of the highlights:
Offset Tracking: telescopes that support this capability can be set to follow slow moving objects. This can be done by entering the R.A. and Dec. rates manually, or by letting SkyTools to it automatically; just target a Comet or Asteroid and the telescope should automatically track its motion. I can't wait to hear from someone how it actually works in the field!
Mount Alignment tool: I have developed a simple, but I think very useful, tool to align your telescope mount via the drift alignment method. SkyTools will pick an appropriate star for you, target your telescope if connected, and display a finder chart. The best part is that the eyepiece view of the chart splits the sky into north and south zones that are labeled by the direction in which you should move your mount. It takes all the guess work and confusion out of the process. I am really happy with it, but again, I'm anxiously awaiting to hear what the testers think.
Native Argo Navis support: in addition to directly supporting the Argo Navis in Pushto mode now you can download SkyTools observing lists to the unit for use in the field. Comet and asteroid orbital elements are automatically uploaded and you will be able to give each list of objects its own identification to help organize your observing.
I have also added support for Pier flipping and selecting basic tracking rates.
Finally, in the process of making charts for finding the exact location of each pole for polar alignment, I realized that SkyTools was somewhat deficient when it came to targeting the poles. So I added a new "object" classification called Reference Points, which includes the Celestial Poles, Galactic poles, galactic center and Zenith. These reference points can be chart targets, added to observing lists, and appear labeled on the charts.
As you can clearly see, I got carried away again.
Before I get to the fun stuff, let me say that "it" has happened again. At every point in the process of developing SkyTools 3 I have run into the very same problem, and even I, head deeply buried in the sand, am beginning to see a trend. I have been operating under a very basic misconception. I know this may sound ludicrous to some of you but I naively thought that SkyTools 3 would be easier than SkyTools 2. My reasoning went something like this: ignoring the database updates, which I know is always an enormous job, most of the actual programming was already in place. To add a new feature or extend functionality would be simple. But that is not the case. Instead, as the program becomes ever larger and more complex, just about anything I do to it becomes more difficult, requiring ever more time and testing. It has become rather clear that the methods I had used up to now, which had served me well, are no longer viable. I have begun to rethink everything with one goal: to release an upgrade every two years. Wish me luck.
Looking at the calendar I can see it is already April and I haven't got the Pro version testing up and running yet. The main reason is that, once again, I have greatly underestimated how much there was to be done. Not to mention all the distractions, like spending several hectic days and late nights last week moving my web site to a more reliable host. And even the testing is proceeding much more slowly than our original Real Time testing did. For instance, I released a bunch of new Real Time features last Friday with the expectation that at least some testers would try them out over the weekend. But so far I have heard nothing. I don't know if it's weather, if the testers have lost interest, or what. Regardless, it is a bit disheartening. I'll have to move on. And when bug reports finally do come in I'll have to go back and figure out what I did, where the code is, remind myself how it all works, and in the process leave what I have started in the meantime half finished. Then I'll have a similar problem when I go back to what I was working on.
But as always, there is a pretty big pot for at the end of this rainbow for SkyTools users. My knack for overreaching means that there are many new features, several of which were unplanned. Here are some of the highlights:
Offset Tracking: telescopes that support this capability can be set to follow slow moving objects. This can be done by entering the R.A. and Dec. rates manually, or by letting SkyTools to it automatically; just target a Comet or Asteroid and the telescope should automatically track its motion. I can't wait to hear from someone how it actually works in the field!
Mount Alignment tool: I have developed a simple, but I think very useful, tool to align your telescope mount via the drift alignment method. SkyTools will pick an appropriate star for you, target your telescope if connected, and display a finder chart. The best part is that the eyepiece view of the chart splits the sky into north and south zones that are labeled by the direction in which you should move your mount. It takes all the guess work and confusion out of the process. I am really happy with it, but again, I'm anxiously awaiting to hear what the testers think.
Native Argo Navis support: in addition to directly supporting the Argo Navis in Pushto mode now you can download SkyTools observing lists to the unit for use in the field. Comet and asteroid orbital elements are automatically uploaded and you will be able to give each list of objects its own identification to help organize your observing.
I have also added support for Pier flipping and selecting basic tracking rates.
Finally, in the process of making charts for finding the exact location of each pole for polar alignment, I realized that SkyTools was somewhat deficient when it came to targeting the poles. So I added a new "object" classification called Reference Points, which includes the Celestial Poles, Galactic poles, galactic center and Zenith. These reference points can be chart targets, added to observing lists, and appear labeled on the charts.
As you can clearly see, I got carried away again.
Thursday, February 7, 2008
Time to Get Real
I am pleased to report that the standard edition test/development is finally winding down. There are still some loose ends (there always are) but the vast majority of the program is now in its final state. Man, it's been a really long haul!
The next step is to begin the Real Time testing. I nearly have all the new SkyTools 3 features integrated into the Real Time tab. In the next day or two I'll release it to the team for basic testing, which I have every reason to expect will go quickly. Then the fun begins! I've sketched out the new mounts and ASCOM features that I want to extend Real Time to support. Primarily that will involve adding support for the Argo Navis, Servo Cat, and various push-to devices based on the Tangent chip. Working in tandem with a test team is essential for this next step since I don't have access to any of these devices. Nevertheless, based on previous experience with Real Time, for once I'm confident that this won't take more than a few weeks.
The next step is to begin the Real Time testing. I nearly have all the new SkyTools 3 features integrated into the Real Time tab. In the next day or two I'll release it to the team for basic testing, which I have every reason to expect will go quickly. Then the fun begins! I've sketched out the new mounts and ASCOM features that I want to extend Real Time to support. Primarily that will involve adding support for the Argo Navis, Servo Cat, and various push-to devices based on the Tangent chip. Working in tandem with a test team is essential for this next step since I don't have access to any of these devices. Nevertheless, based on previous experience with Real Time, for once I'm confident that this won't take more than a few weeks.
Subscribe to:
Posts (Atom)
