actorartathleteauthorbizcrimecrosspostcustomerservicedirectoredufoodgaminghealthjournalistmedicalmilmodpostmunimusicnewsworthynonprofitotherphilpolretailscispecialisedspecializedtechtourismtravelunique

IamA Software Engineer who worked on Juno (satellite going into Jupiter orbit today) AMA!

Jul 4th 2016 by pennstatephil • 22 Questions • 255 Points

My short bio: I was a space nerd growing up, and my 2nd assignment out of undergraduate was a dream come true: working on the fault protection systems for the Juno and GRAIL spacecraft (which were being developed simultaneously). I also worked on testing the software for the missions once development was complete.

Today, almost 5 years after I watched in awe of my work roaring away from a launch pad in Florida, Juno has finally arrived at Jupiter and is ready to begin its mission! Watch JOI (Jupiter Orbit Insertion) coverage on NASA TV.

If I can't answer your question (especially if it's about the science-y stuff), JPL has a great press kit containing lots of detailed information about the mission and the science it'll be doing. A couple flight engineers also did an AMA a few days ago, and your question may have been asked and answered there.

Finally, the mission has its own subreddit- check it out at /r/junomission (although I'm sure /r/space will give it plenty of attention, too).

My Proof: http://imgur.com/a/ilZ9D - note some of the proof is GRAIL related.

  • 8:42 Eastern - gonna take a break for dinner and enjoy some of the holiday with friends, but I'll be back later to answer more questions. I'm having a great time, and thank you all for the questions and being excited about space!
  • 10:20 Eastern - I'm back-- less than an hour to go to JOI! IT'S HAPPENING!
  • 11:55 Eastern - BURN COMPLETE! WE DID IT! I am extremely happy/relieved. I'm going to have an adult beverage or three to celebrate. Thanks for participating, everyone. I'm happy to keep answering questions, but I'm gonna call it a night for the AMA.
Q:

What software and coding languages do you primarily work with? I'm attending college to become a software engineer right now and would love to work for NASA or spacex one day

A:

Great question! We developed the flight software in C++, and the drivers in C. The flight software was well established and flown on multiple missions before Juno/GRAIL, so it was more customizing the existing system for our mission. If you're looking to work on embedded systems, C++ is a great place to start. I also have done a lot of work in Java since. I sincerely believe that as long as you learn good fundamentals, you can learn any syntax.


Q:

Thank you! I've been working with Java for a year now and have started to dabble around with c# and Python. Additionally I also like to work with html, Css and JavaScript although web development isn't what I would like to pursue. Hopefully one day I can do what you do. It is my dream after all. Thank you for your time and response. Good luck with your future work.

A:

Good luck to you too!


Q:

What is your favorite language?

A:

It's hard to say-- every language has a purpose. I'm a big fan of Java and C# for back-end projects, but I've used some really slick javascript libraries for front-end stuff.


Q:

How accurate are the statistics on salary earnings for computer engineers as of 2017, according to you?

A:

Depends on the source. It really comes down to experience and where you live. Denver's a pretty hot tech market right now.


Q:

Any tips for a person that would want to become a software engineer?

A:

Start as soon as you can, and always be ready for change. I've used lots of different languages throughout my career, and some have worked dramatically different than others. So, whatever you learn, make sure you're learning good core values rather than the nitty-gritty stuff. Buy lots of books, always be reading up on new technologies.


Q:

I'm going to be majoring in software engineering starting this fall. Any tips that will help me along the way?

A:

Don't be afraid to ask for extra help on things you don't understand. I learned some of my best stuff at office hours and study sessions. Don't sweat the syntax as much as the core concepts and data structures. Syntax will change, but "if"s and "loop"s are in every language.


Q:

Hi! I'm currently finishing my degrees in Comp Sci and Mathematics and I was wondering: how much domain knowledge do you need to get into an industry? If I wanted to work for nasa or spacex, would I need any sort of degree in physics?

A:

Nope. Code is code. You learn the business as you're in the industry. If you're passionate about it, all the better, but you certainly don't need a degree in astrophysics to work on a spacecraft.


Q:

We are!

I'm at psu as well (Mech Eng '17). I will be interning at nasa's JSC this fall. Could you go into the pros/cons of working for NASA vs. one of their contractors?

Thanks for doing this AMA. Good luck tonight.

A:

Unfortunately, I can't, really. I've only ever worked as a government contractor, so I'm not really sure what the differences are. I think probably the main advantage is you're less constrained by all the lovely government workplace regulations that are in place.

Enjoy the internship, and I'm always happy to help out fellow PSUers-- PM me if I can help!


Q:

Hey, thanks for doing this AMA!

I have a general question. Lately I've noticed that a lot of people who major in some form of engineering, like biomedical, or in your case software, are working in fields way out of their comfort zone. Working on a space mission doesn't seem like the kind of job I'd expect from a software engineer (which, by the way, is absolutely amazing!).

How broad are my options when majoring in a certain field of engineering? Do I have plenty of options, rather than just a few like I've always thought?

A:

The beauty of software engineering is that everything uses computers now. You can be developing websites, educational software, medical software, ridesharing apps... where there's a computer, there's a need for someone to code it. I was very fortunate that my interests aligned with the opportunities I had. I think it's very rewarding that I can jump from industry to industry and learn more about each one, while still speaking the common language of software.

The best way to figure out what jobs are available for a field is by googling it. If you're gonna be a petroleum engineer, search for petroleum engineering jobs. Same for mechanical, computer, biomedical... a base knowledge of these fields can be applied in many industries.


Q:

Is there any backup equipment or redundancies in place in the event that something goes wrong or stops working, or is the available space and weight restriction too limiting for such things?

A:

There are very many backups and redundancies. The spacecraft is "two sided", there's redundant hardware on each side, and 2 copies of the flight software on each side (4 total). The system is constantly running checksums since Jupiter's magnetic field is so strong, and the possibilities of bit flips is very high. If a copy doesn't pass the checksum, it's marked bad, and overwritten with a good copy.

If hardware or software fails on one site, it autonomously reports the fault and fails over to the other side. This is primarily the software I worked on (the detection and handling of faults). There are also redundant paths, for example an "A" and "B" path to the communications arrays, so if a path fails, it's marked bad, and fails over to the alternate path.

Plus the whole thing is in a titanium vault :)


Q:

IMHO this is the most interesting reply from the OP in this thread! Is some sort of similar redundancy used as well for the science data as well as the flight software? I assume there is typically a large amount of data buffered on the craft before it's able to be piped down through DSN.

Can you / are you allowed to say whether the spacecraft is running some well known embedded OS such as Wind River or QNX?

A:

I'm not as sure on the science data, but I do know that after JOI, the spacecraft will be pointed towards the earth (and the sun, for power) as much as possible, so data will be sent down quite often. It would make sense to make multiple copies of the data and run checksums on it, again, due to the potential magnetic interference.

I don't believe it's running a traditional OS (I honestly don't remember), it's bootstrapped to just start up and run the flight software.


Q:

Do you think we'll be able to use the information we are learning from Hubble and Juno about solar wind affects/auroras on Jupiter to better understand the possible impact of changes past/present/future with-in Earth's atmosphere?

A:

disclaimer, it's not my field; but yes, I do think some of the specific experiments should tell us much more about the atmosphere in Jupiter, and since the largest planets form first in a solar system, we should get a good idea of how all the other planets formed.


Q:

largest planets form first in a solar system

Why is this?

A:

The short answer is "Gravity." The current accepted theory is that planets formed via stuff sticking together and being pulled into each other by gravity. Bigger bodies attract more stuff, making them even bigger, attracting more stuff, and so on. So, the biggest planets have been around for the longest time, pulling together the most stuff. See https://en.wikipedia.org/wiki/Formation_and_evolution_of_the_Solar_System


Q:

How much do you get paid?

A:

I won't give exact numbers, but I made a lot more when I left government contracting. Cool jobs will always be long hours at not super good pay, because lots of people want to do it, so you're totally replaceable with some other person who will accept it. I didn't get a pay increase for the 2.5 years I worked on this project.


Q:

Hi there, thanks for doing this AMA! First off, congratulations on your accomplishments and I hope this mission turns up some very interesting answers for you.

I just recently graduated with a BS in Computer Science and have been looking into the job market with no luck. I would love to work for NASA or a similar agency that is involved in space-related technologies, but am not entirely sure where to even begin besides applying directly. Unfortunately, college internships have passed for me but do you have any advice on a possible plan or route I should take? I have heard some people suggest that I try to get a few years of industry experience first, but I'm not entirely sure.

Also, I have an obligatory question. Do you think Pluto should be a planet or not? :) Thank you so much!

A:

Thanks for the questions and the congratulations. I'll feel much better once Juno's safely in orbit.

NASA does not build many things itself, as far as I understand. Most of the work is contracted out to government contractors. I worked for Lockheed Martin, but there are plenty of other contractors (Raytheon, Northrop Grumman, etc) and local ones (some out here are ASI, Red Canyon, and more).

If you can't get directly onto a mission, it's easier to get your foot in the door and then transfer.

As for Pluto, I gotta side with NDT on this one. Dwarf planet.


Q:

Thanks for the reply! I recently got a call from a Boeing recruiter for an entry-level software engineer position so hopefully that can open up some doors for me in that area.

Best of luck with your future endeavors and hopefully we learn a lot of great things from Juno!

A:

Boeing does some very cool stuff, good luck!


Q:

What is the hardware platform on Juno, for your C++ / C to run on? Is it some derivative of x86 (rad hardened), or a totally different architecture?

What OS are you running?

A:

I wish I remembered more about the hardware, but it was a long time ago and I didn't have much exposure to it. I do believe it was x86, and there isn't really an OS to speak of, it's just "turn on, run program"-- that part might be unix based. We definitely developed on unix machines. So no OSX or Windows :P


Q:

I'm very jealous, what was it like working on Ellen Page?

A:

She's built like a steakhouse, but she handles like a bistro!


Q:

Thanks for doing this AMA! I have a question about bugs.

I'm a web developer, and I create bugs all the time. My code isn't bad, but I work fast and refactor later, etc etc.

Is there a concept of "tech debt" in your line of work? How extensive is the testing? Do things just take a really, really long time to build?

A:

Now, things may have changed in the past 5 years, but when I was there it was very much waterfall. Requirements->Design->Code->Test. Every requirement had to have a test verifying it. We would also discover more bugs in integration, and fix those. It was very VERY well documented. And yes, things generally just take a really long time to build :)


Q:

SWE here as well (I don't work on space stuff though). This is awesome. Can you say more about what the testing frameworks were like? I'd imagine unit/integration testing is really insanely important to you guys.

What is the flight software stack like? What platform are the drivers written for? How do you do software updates above the cloud?

How do you deal with communication delay to the spacecraft? Do you have to keep a tmux session open like we do here on earth when our wifi gets shitty?

Okay, that last one was a joke. Mostly.

A:

Every piece of code was unit tested, then we integration tested the spacecraft in many different scenarios. Every requirement is verified by a test.

The FSW has many subsystems (thermal, communications, fault protection, etc), and they all work in congress together. Software updates are sent as patches, which are loaded, checked for validity, applied, and then the spacecraft restarts.

The spacecraft is very autonomous, so we tend to be pretty "hands off" when trying to drive the spacecraft. We send very broad commands, and the spacecraft knows how to do the rest :)


Q:

What sort of testing is done on this sort of software? Im sure it's extensive.

Are you still nervous you missed something and a null parameter exception is going to cause complete mission failure? :P

A:

I am dreading that very thing right now. Funny story, the software was... exercised... unexpectedly when going around earth. Someone had applied the wrong parameters and the power level dropped when it went into earth's shadow, so the spacecraft went into space mode. This is the correct behavior, but before they had found out the root cause, I was beside myself thinking I screwed something up.


Q:

Hey Phil! Congrats on today, I bet it must be very exciting to see this take place.

What kind of scenarios were used in testing the software, and how long does one QA a piece of software for such a complex mission like Juno?

A:

Thanks!

There were a bunch of mission phases; launch, inner cruise, outer cruise, JOI (now), and science. Each critical path was tested in each mission phase. I'd say at least a dozen of us worked on testing for many months (at least 3). It took a lot of work and lab time.