Combining 15s interval whole-sky-camera photos to form a 4y spanning keogram
astrodon.social

・

528 points

・

nebalee

・

7 days ago

92 comments

isoprophlex ・ 7 days ago

This is fantastic, and like another commenter mentions, a beautiful continuation of an age old practice: measuring the sky.

Very cool that their hardware keept chugging along for years without hiccup, too.

If you want to do something kinda similar but far less involved: a very lo-fi, no computer involved thing to do is an ultralong photographic exposure (months, a year, longer) with a pinhole camera.

The results are quite artistic IMO [1], the camera is fire-and-forget and you don't need any chemicals to develop the image. Just photograph/scan the photographic paper and invert the colors.

I'm not affiliated with them, but Solarcan sells ready made single-use pinhole cameras. An almost zero-regret purchase I'd say.

[1] You see the sun move through one year of skies, as seen from my balcony: https://files.rombouts.email/IMG_6500.jpeg

People have made wonderful, mildly spooky pictures with these: https://solarcan.co.uk/wp-content/uploads/2018/05/solarcan-p...

Tepix ・ 7 days ago

https://archive.ph/usfd5

To avoid HN hug of death on this 1832 users Mastodon instance.

Very cool arrangement of those pictures, i was wondering what he has done about the daytime pictures when i read the title.

crazygringo ・ 7 days ago

Incredibly cool. I love this so much, both artistically as well as how it demonstrate the equation of time [1] -- the fact that the changes in sunrise and sunset are not symmetrical.

On the other hand, it's driving me absolutely crazy that he centers the image at 4:00 rather than midnight. Or maybe that's to show the shimmer of sunlight a little after noon on the right hand side?

I can't figure out why it's "bluest" closest to dawn and dusk. I'm guessing the exposure makes a huge difference, and obviously the night part is way more exposed than the daylight part, or else it would be much darker. Wondering if the camera used automatic exposure, and how much of the brightness of colors in the image are artifacts because of that? Also if he locked the white point hopefully?

[1] https://en.wikipedia.org/wiki/Equation_of_time

  • bee_rider ・ 7 days ago

    One of the commenters noticed this as well, the author says it is something automatic in the exposure.

    https://fediscience.org/@Birk_lab/113770845539931892

    • crazygringo ・ 7 days ago

      Oh that's a real shame then. The resulting composite images are certainly artistically interesting to look at, and you can see the big-picture effects like sunrise/sunset and moon, but that explains why you can't see the gradual brightening at dawn, or degrees of darkness at night.

      It seems like if you wanted to do this accurately, you'd need to lock exposure to handle a bright blue sky without blowing out -- both aperture and shutter speed. And lock white balance. The question is whether that would allow for sufficient sensitivity at night. But if you're just averaging color values across a section of sky and mainly looking for moon and moonlit clouds, I think it would, since pixel noise will get averaged out and the moon is bright.

      • jupiterelastica ・ 7 days ago

        Additionally to locking the exposrue time and aperture, one could also take multiple exposures, figure out the camera's light response function and fuse multiple exposures together into a single higher dynamic range (HDR) image (see OpenCV tutorial on that or Debevec et al. 1997) Assuming you can find the camera response for the very long exposure times at night _and_ the very short during the day, one could relate them to each other and display both for accurate visual comparison.

      • dekhn ・ 7 days ago
        ・ 2 more

        Yeah, I agree this should be using a fixed exposure (possibly on a schedule) with locked white balance. They are using an astro camera so the sensor is very sensitive, they can get away with extremely low exposure times.

        • teamonkey ・ 7 days ago

          > get away with extremely low exposure times

          For night sky photography with an astro cam you’re still looking at exposure times of 20-60s at night (possibly also increasing the gain at night) and milliseconds during the day. The dynamic range is immense.

          As someone who has struggled with this for my own allskycam, it’s extremly difficult to have white balance settings that perform well at all times of the cycle, especially with a camera designed to be more sensitive in the IR part of the spectrum (which will always look unrealistic). Settings that give you lovely white clouds and blue skies during the day tend to give you purple skies and green clouds at night. The quality of light is different so the white balance is different.

          You can use autobalance or different white balance profiles for day and night but they each have issues.

    • sneak ・ 7 days ago

      #1 rule of timelapse: manual white balance, manual shutter speed, manual aperture, manual agc/iso, manual focus.

  • brookst ・ 7 days ago

    I think the blueness is because clouds at dawn and dusk reflect atmospheric colors more, whereas midday the clouds light up as more of a white from diffusing sunlight. 100% made up theory on my part, but I think it makes sense?

    I asked o1 to estimate colors by hour and its reasoning and estimates seem fairly convincing[1], and also show more saturated blues dawn and dusk, though it did not model clouds.

    1. https://chatgpt.com/share/67795fe3-9ac8-8009-9922-153f40c509...

    • teamonkey ・ 7 days ago

      Not sure about the accuracy of the results but the theory’s correct. The colour of sunlight changes as it passes through different thicknesses of the atmosphere and the proportion of direct light, refracted light and bounced light changes.

  • fsckboy ・ 6 days ago

    >he centers the image at 4:00[pm]

    I think because the night sky offers interesting variation that looks nice stitched together. the sunny day, not so much. I will also point out, OCD notwithstanding, that 0hour/12am is an arbitrary time to start the day, and you should really look at the days/nights all strung together in a long continuous strip

tppiotrowski ・ 7 days ago

I generate these type of charts [1] focused on the daylight hours so it was a surprise to see a concave shape instead of a convex one. Awesome way to validate these computer generated charts with captured physical data.

[1] https://shademap.app/@52.39941,4.88468,11.49849z,17360064872...

  • nozzlegear ・ 7 days ago

    This is really neat. I'm curious where the data for the tree shadows comes from though. I was surprised to see that the trees in my yard and my neighbor's yard were all mapped by your service, since I live in a small town in the middle of nowhere. I read the "how it works" FAQ section, which explained that building shadows come from the map services, but it didn't mention trees.

    • isoprophlex ・ 7 days ago

      I built a similar shadow mapping tool for some commercial party that wanted to accurately estimate solar panel production in The Netherlands... In my specific case I could access very accurate LIDAR heightmaps gathered from planes.

      This means you can ray-march the location of the sun throughout the year over the entire country to calculate exactly where and when a surface is occluded by shadows from nearby (or even faraway, sometimes) objects.

      The LIDAR data can be as detailed as a shadow cast by antennas, a chimney or a tree... Which is more important than you'd think, because a little bit of shadow on a single panel means that all panels daisy-chained to that panel will see an efficiency drop! (So you either don't chain them but give each panel its own inverter, or you wreck your neighbors chimney)

fiforpg ・ 7 days ago

Annnd, this is exactly how humanity figured out astronomy before the telescopes: by taking multi-year observations and keeping detailed records! See Antikythera mechanism, Tycho Brahe, etc. Arguably the entire Greek epistemology was so solid because their astronomy turned out to be an early proving ground for the scientific method.

  • dylan604 ・ 7 days ago

    This is something I mention to the younger kids at start parties. Way back then, there was no internet, no devices, no tv, no radio, no movies, and more importantly no easily accessible calendars. You couldn't help but look up at the night sky. There was really nothing else to do, and when the only thing available to know when to plant and harvest was based on the constellations visible at the time. It helps put things into a way that helps them think about it even if they can't quite comprehend not having devices.

    • crazygringo ・ 7 days ago

      > There was really nothing else to do

      Well, nothing to do except for storytelling, music, dancing, parties, games, plays, sports, eating, drinking, visiting, festivals, gossip...

      People have been pretty good at keeping themselves entertained for a very long time.

      • arduinomancer ・ 6 days ago
        ・ 3 more

        Obviously OP didn’t mean literally nothing else to do

        • crazygringo ・ 6 days ago
          ・ 2 more

          The reason I bothered to respond is that there is a common tendency to look back at historical people and see them as extremely primitive -- with little knowledge, with little entertainment, with little going on. That they would have had nothing better to do than to look at the sky.

          And that's what I'm pushing back on. No, that's just not true. Obviously some people were interested in the sky, and it was a tool for certain things, but OP literally said "There was really nothing else to do." That's a complete misunderstanding of human society in history. And it's important we don't reduce our ancestors to these primitive stereotypes.

          It's also important to push back on the idea that there were no distractions. OP says "no internet, no devices, no tv..." but there was extensive village gossip, games, stories... fundamentally we had all the same kind of distractions. We're a lot more similar than OP thinks.

          • dylan604 ・ 6 days ago

            You're now assuming that I'm a bumbling idiot that thinks everyone before electricity were neanderthal cavemen. Although, I've met plenty of modern people that couldn't hold a flame to a caveman.

            The fact that the ancients knew the coming of the seasons and developed their entire society based on what was visible in the heavens just shows how much they did pay attention to it.

            There's a thing in story telling called hyperbole. It helps enhance the story by bending the truth just a bit to make the story more interesting and helps get a point across. It is especially useful in oral story telling.

            But clearly, we're all bumbling idiots for not getting the minute details exactly right. Good to know. I'm glad you're so intelligent that you've set me straight

      • globular-toast ・ 6 days ago

        I guess he meant there was nothing else for nerds to do...

      • dylan604 ・ 7 days ago
        ・ 5 more

        And exactly how does that help the specific narrative?

        • crazygringo ・ 7 days ago
          ・ 4 more

          You were writing as if people were staring at the sky all night long out of boredom. You said "There was really nothing else to do."

          That's completely untrue. It's a total misunderstanding of what life was like.

          I'm not helping your narrative, I'm saying it's a false narrative. We have so many misunderstandings about how people used to live, it's important not to perpetuate them.

          • dylan604 ・ 7 days ago
            ・ 3 more

            Everything you listed as options to do are also available now, yet people still choose to stay home and look at their devices. Pedantry for pedantry sake is just boring

            • crazygringo ・ 6 days ago
              ・ 2 more

              > Everything you listed as options to do are also available now

              That has no bearing on anything you or I said.

              > Pedantry for pedantry sake is just boring

              Perhaps you don't understand what pedantry is? Your entire point was a false description of what humanity was like. Just take this as a learning experience! :)

              • dylan604 ・ 6 days ago

                Perhaps you don't know what story telling is. Just because details were left out does not mean I'm implying they didn't happen. It's not a lie of omission. You're hell bent on making something out of nothing, for what point I have no idea. But you're free to continue to attempt to denigrate my intentions if that really makes you feel superior.

                You've yet to dispute the actual point that the average person did not have a simple device that allowed them to see the future without any thought about what it took for that to happen. In order to know how much of a season was left was to look at the night sky to see what stars where visible. Not the day time sky. The only reason they knew what they knew because people very intently looked at the night sky and started to recognize the patterns. Their was no Hey Google! What is today's date? or Hey Siri, how many more moons until it's time to plant seeds?

                But sure, let's focus on the irrelevant parts of whatever story you want me to tell. In fact, how about you go find a star party and volunteer your time to talk to kids and tell the story however you want. Then come back and we can all critique it to ensure you didn't leave out any irrelevant details. Maybe we should update the saying "those that can't do, teach" to add "those that can't teach, critique"

    • morsch ・ 6 days ago

      Like the sibling, I don't think people had trouble entertaining and distracting themselves. But there was no electrical light, and much less artificial lighting overall. Evenings and nights were dark. I'm sure many a walk home after a gathering was lit only by the night sky.

      Meanwhile, I can see the moon and like ten stars on a typical night, because I live in a city and there is massive ambient light at night.

      • dylan604 ・ 6 days ago

        I initially had something referencing light pollution, but felt it wasn't as relevant to the point and took it out. When talking to kids, giving them the whole nine yards at every opportunity is overwhelming. Giving them just enough to get the point across is much more effective. Explain once without over explaining, demonstrate, and then let them experience it. Never been a teacher, but it works as a coaching technique that translates well in other areas. Giving too much info that's not relevant verges on mansplaining IMO.

drivers99 ・ 7 days ago

I was confused why the night gets so short (looks like 1/6 of the time) but I looked it up on a site that shows sunrise and sunset times along with twilight and in the summer it doesn’t even fully get out of twilight. And it seems that there is no correspondingly short day because the sun takes precedence in that the light can bleed over into the night after sunset / before sunrise but not vice versa.

https://www.timeanddate.com/sun/netherlands/amsterdam

hinkley ・ 7 days ago

My favorite work in this vein is still the guy who did the trigonometry to convert a year+ worth of telemetry from a light level sensor in his back yard to paint a picture of the tree canopy. Each day is a scan line of light and dark patches at a different solar inclination. So everything above the sensor eventually got “painted”.

  • azepoi ・ 7 days ago

    Have you got a link or do you remember the name of this person?

    • hinkley ・ 6 days ago

      Oh gosh it's been a while. I thought it might have been in r/dataisbeautiful but I'm coming up with nothing. I have a vague recollection of what the picture looked like but search isn't turning it up.

imoreno ・ 6 days ago

This is neat, but the amount of data seems overkill just to see the trend in day/night length and the keogram is a bit busy. It seems like compressing each day to 1 px tall would result in a clearer representation of that.

It's neat that you can look at different things, like cloud cover, the moon, stars and anomalies (such as the aurora). But these are convoluted in the raw data. Deconvoluting these to obtain a keogram of just the sun, just the moon, just the anomalies would probably make for some interesting visualizations.

acegopher ・ 7 days ago

I did much the same thing (and much more) 10 years ago. I used a RPi as well as a Canon point-and-shot out my window. My pics were 1 per minute.

Here is a link to the keogram section (I didn't know the term), but watch the whole video, I did a bunch of other art:

https://youtu.be/Mfo4hVc71Qw?si=3YKojggkTj2xehAB&t=2334

  • kaeland ・ 6 days ago

    I really enjoyed your faces example! I’ll have to look at the rest of the video soon, but it’s so cool what you can do with a little imagination and ingenuity!

gregfjohnson ・ 6 days ago

This brings to mind something I've wondered about for a while. Sunrise on the shortest day of the year is earlier than sunrise for several days after it.

Sunset on these days after the shortest day is of course even later than sunset of the shortest day.

On the beautiful image of the OP, you can see that after dawn of December 21, dawn continues to get later over the next few days.

In my area, sunrise on 12/21/2024 was 6:54am, and it will continue to get later until 1/8/2025, when it is at 6:59am.

Length of day on 12/21/2024 is 9 hours, 54 minutes, and length of day on 1/8/2025 is 10 hours, 2 minutes.

Searching the web, I haven't found an explanation for this that "clicks" for me as both intuitive and rigorous. Any thoughts or pointers on this?

  • gregfjohnson ・ 5 days ago

    OK here is my attempt at intuition:

    Here is an approximation that captures the main effect (the 23.5 degree tilt of the earth's rotation axis) and overlooks secondary effects.

    Consider the equator. Imagine a circle on the X-Y plane centered at the origin. Angle the circle up 23.5 degrees, rotated around the x axis. The projection of this circle back onto the plane is an ellipse on the X-Y plane, with the vertical axis about 92% of the length of the horizontal axis. Now, consider a series of vectors in the X-Y plane starting on the X axis, with angles in steps of 0.986 degrees. (This is approximately the angle the earth progresses around the sun each day.)

    Where each vector hits the unit circle, move the point up or down so that it hits the ellipse. The angle will change a bit for most of the rays. In some cases the angle will be a bit smaller, and in some cases a bit larger. These discrepancies are the variations in time of day of sunrise and sunset over the course of a year on the equator.

  • incognito124 ・ 6 days ago

    The apparent movement of the sun is not influenced solely by the Earth's rotation, but also the instantenuous velocity of revolution and also the fact that Earth's axis is slanted w.r.t. the ecliptic.

    Study this Wiki article, especially the components part: https://en.wikipedia.org/wiki/Equation_of_time

  • KineticLensman ・ 6 days ago

    It’s partly because we have a standardised 24 hour clock and solar noon (when the sun is highest) is sometimes ahead and sometimes behind GMT noon. The sunrise and sunset times relate to solar noon so they vary accordingly.

    See ‘equation of time’ and ‘analemma’ for underlying astronomical explanations, as hinted at by sibling posts.

  • gregfjohnson ・ 6 days ago

    Thanks, very much appreciate the thoughts and pointers!

kbutler ・ 7 days ago

I was surprised there wasn't more of a line of red from sunrise/sunset.

It's probably because the composition algorithm takes a central line through each image (a line through the zenith), so it captures relatively few horizon-adjacent pixels that would highlight the reds of sunrise/sunset.

qwertox ・ 7 days ago

It took me a bit to understand that it's just the red line from the entire dome. It makes me a bit sad that so much information is discarded for these keograms. Isn't there a way to "peel" the image in such a way that the entire image ends up being the line? Like spiraling or something like that from the outer "rim" to the center point.

Nevertheless, it's the first time I've seen this and liked the project a lot. I've seen this from normal images, but not from such a fisheye lens.

Really cool project.

Edit:

Looking at https://victoriaweather.ca/keogram.php?photo=20120810.jpg how can this contain the entire landscape if only the center line is used, which is supposedly always the same line? I mean, the camera isn't rotating. Is this just another kind of view generated from the dataset?

  • gmiller123456 ・ 7 days ago

    The goal of the keogram is to give a quick overview of sky conditions so you can see if there were clouds, aurora, or other interesting activity. No information is discarded, as the user keeps all of the data, the keogram is just a way of identifying which pictures might have something interesting.

    The year long keogram presents even less data, as it's just the centerline of the keogram for each day. So, essentially just the center pixel of each image. Still gives a good overview of what the sky conditions were like throughout the year.

  • zerocrates ・ 7 days ago

    The landscape picture there in your link is a different kind of thing; that one has the column used from each individual photo/frame advance from left to right through the day when constructing the final image, so it still looks like the same static view of the landscape. Of course it's also just a different camera view entirely as well.

    The fixed column (and upward view) approach used in the main link is better for showing the movement of the sun/moon/stars.

  • thanatos519 ・ 7 days ago

    I want a movie made from rotating that red line.

  • jstanley ・ 7 days ago

    But there are lots of images throughout the day.

metadat ・ 7 days ago

Really cool, this is the first time I've heard of a keogram.

I wish there were a corresponding article with a technical write-up covering both the capturing rig and image assembly process.

Is there an FFmpeg or imagemagick command, or perhaps a Photoshop script to feed the images into?

  • dekhn ・ 7 days ago

    The capturing rig itself is pretty simple. It's a straightforward project box you can get from amazon. The camera is an astro camera, but you don't need that- a straightforward USB camera from Arducam would work fine (it has to be a wide angle or accept wide angle lenses. Or a raspberry pi High quality camera. That plugs into a raspberry pi which runs some sort of script or cron job to take a still image. There are CLI tools to do this as well as modify the camera settings (exposure, white balance, etc). There are some holes drilled in the case with a simple plexi dome. Note inside the dome is a ring of resistors which are given electricty through a relay (which is under software control by the pi) to heat up and keep the enclosure clear. I also see a pressure/humidity sensor (likely used to control the relay) and a fan and some desiccant to help keep it dry.

    For the software processing you can use anything; I would write a python script to create a large canvas, and then downscale the images and place them at the appropriate location. You could also load the whole dataset into a large tensor and do anything you want with it.

gmiller123456 ・ 7 days ago

You've got much better weather than we do. Here's mine from near Louisville KY (from Aug 2023 to Aug 2024) https://www.facebook.com/groups/101640086556073/posts/794426...

  • russellbeattie ・ 7 days ago

    It's easy to forget how far north Europe is! The daytime vs nighttime hours is so much more extreme there. The Netherlands is on the same latitude as Newfoundland in Canada.

    I noticed the extreme hour glass shape in his image, but it's even more impressive when compared to yours!

    • incognito124 ・ 6 days ago

      I always forget how south the US is. Rome is approximately at a similar latitude like Boston!

bikamonki ・ 7 days ago

So cool that the shape resembles an hourglass!

  • mjbrownie ・ 6 days ago

    I work with a shorebird expert who has been tracking migrations (Latham snipe whose range extends from Russia/Northern Japan to Southern Australia).

    I think an early technique of tracking their migration before gps/chip/batteries were small enough was a primitive light sensor + data logger that would log day/night hours on the bird as it flew from island to island and the light data was enough to roughly estimate lattitute (and often enough to infer a location/date.

    The data looked great in this form. It wasn't hourglass but with jagged edges and shifts. You could easily see which months the bird stayed in a single place for several days etc. clever stuff.

    • tanseydavid ・ 5 days ago

      Snipe, huh? Will you be organizing a Snipe-Hunt any time soon?

      • mjbrownie ・ 4 days ago

        nice. had no idea about that connnotation :) ... goes with black swan event as a saying that's ill fitting for these parts (in southern Australia all we have are black swans).

diyseguy ・ 7 days ago

Why does the sun shrink and disappear just past the halfway point on the keogram?

  • icehawk ・ 7 days ago

    This post? https://astrodon.social/@cgbassa/113770444896606353

    That's the moon, and it looks like it's increasing cloud cover + dawn causing camera to decrease overall exposure.

    • diyseguy ・ 3 days ago

      I don't buy it. That's clearly the sun until it reaches that red line and then it shrinks and disappears. I've never seen anything like that happen in the sky. Moon or sun. It's super weird.

      It's kind of a fun synchronicity though as just after seeing that video, I watched episode 2 of 3 Body Problem where the same sort of thing happens!

j7ake ・ 7 days ago

Would love to have these figures as big posters as art decorations in my house

avipars ・ 4 days ago

Were you able to connect IR LEDs to an official Pi NOIR camera and which ones do you recommend?

krm01 ・ 6 days ago

Wow this is really beautiful. Is there an algorithm (not api, but available calc method) that can calculate the full year based on a few variables? Like long+lat?

leobg ・ 6 days ago

The “stripes” in the keogram look very regular. A few days of clear skies followed by a few days of clouds. Is that what’s happening? Why the regularity?

rollulus ・ 6 days ago

Could someone explain to me why the sun-rise/set doesn’t seem as sinusoidal as I would naively expect?

justinl33 ・ 6 days ago

the real technical achievement here is maintaining consistent camera sensitivity over 4 years. Most consumer CMOS sensors degrade significantly from UV exposure over that timeframe, which would show up as a gradual darkening in the composite

elintknower ・ 7 days ago

Wonder if they're going to open source the code they used!

xeckr ・ 6 days ago

The dress, anyone?

kyrofa ・ 7 days ago

[flagged]

  • Evidlo ・ 7 days ago

    The modal at the bottom says rate limit exceeded. Don't know why it says 404 instead of 429

    • hu3 ・ 7 days ago

      There's no modal at the bottom on mobile for me.

      So just 404.

      And why would someone flag that users message for stating a fact.

      People will click the link, see 404, and leave confused. That flagged message is useful.

  • galleywest200 ・ 7 days ago

    "Rate limited, please try again after 7:45 AM"

    Okay, will do.