Octoprint debugging and logging

3D Fabrication
1

Hello, all. We’ve been listening to everybody’s comments and complaints about the new Octoprint setup, and I’m trying to track down the source of the “Unauthorized” error. To that end, I’ve switched Far Lefty to the development branch of Octoprint and enabled verbose logging for it. If you happen to encounter an error of any kind, please let me know:

  • The time/date of the error
  • The message / symptoms
  • (If you solve the issue) what fixed it

Ideally, if it is consistently producing the same error, it would be helpful if you can leave the printer in the error state and use a different printer. I know this isn’t always practical, but if it’s a slow day it would be helpful in nailing down these issues. I’m part of the 3D printer support email list (the address written on the printers). Shoot a message there as always if there are any issues.

2 Likes
2

Hey! Though I’d pop in to provide info of what I know about the problem, having seen it myself twice in two hours.

  • It changes within the hour - one time you get “Unauthorized”, the next hour it’s fine.
  • It depends on the G-code and computer you use - had Far Lefty refuse to print on anything except the touchscreen monitor. Applied a different G-code to Far Lefty on a different computer after the 17 minute print job had finished and it ran fine.
  • This does mean that if you get the error “live”, you can continue to get the error for an hour or so before you need to find another way to trigger it.
  • Disconnecting and reconnecting Octoprint to the printer itself sometimes fixes the issue, but not all the time.
3

Sean (@zootboy) 'n all… I’ve been printing on Far Lefty overnight and noticed that the “Pause” and “Cancel” buttons are not responding (ONLY on Far Lefty - which is running a DEV version). I also had a possibly related issue with the temperature controls, as I was trying to cancel a job; but, it wouldn’t pause/cancel, so I tried to set the temps to “0”; but, they continued to rise. The printer was still connected (wasn’t getting any of the connection errors). Pressing the red button in this instance did not help. I had to power off the printer with the power switch on the back, then turn it back on - before the temps would stop rising. I’ll send an e-mail to [email protected], too; but, since the “Pause” and “Cancel” are not working at all for me, I wanted to be sure others who might use Far Lefty while it’s loaded with the DEV version, that they will not be able to “Pause” or “Cancel” their print jobs for now…

4

Hi David (@commandhat)…

Thanks for the input! All info we can receive will help us narrow down what’s going on.

It seems to be pretty much the very same thing that would happen with Pronterface - just with different error messages. Back then, when “something” would cause a hiccup/disconnection, there would be no way to do anything with the printer - until after Pronterface was closed and the printer was reset (using the red reset button inside the printer). With Pronterface, there were no obvious “error messages” - because the application would just freeze up (so there could have been an error message trying to display; but, since the app would become totally unresponsive, there would have been no way to know for sure…). Again, when that would happen, Pronterface would have to be closed and the printer reset, then things would be OK again for a while.

NOW - instead of the app totally freezing up like it would with Pronterface, we get actual error messages (the connection, time out, not authorized, etc… errors are all variations of the same thing - just pop up with different wording - depending upon what you are doing at the time).

As with Pronterface freezing up, the error messages we see in OctoPrint will often eventually clear themselves - after several minutes or more; but, not always. If you don’t want to wait so long (and who does? :wink: ), then what “usually” works is pressing the red reset button inside the printer, waiting a couple/few minutes, then refreshing the browser and trying to reconnect. Every once in a while the printer will continue to not respond; and, when that happens, you may have to reach around to the back of the printer to press the power switch (toggle button next to the power cable) to power cycle the printer. Please only do that as a last resort, as I have seen the USB cables get knocked loose - from the back of the printers and from the Banana Pi devices - which cause additional issues, as you can imagine.

Oh and don’t forget to send detailed reports to [email protected] any time you have any break/fix type problems! There is also a log book to log the issue, as well as sheets of paper that should be taped to the printer, if it’s something that will cause others not to be able to use the printer (such as if something gets broken and the printer is truly out of service, etc…).

Thankfully we have a pretty awesome support group - including PolyPrinter (@edkman) who is working with us to ensure there are no compatibility issues within the firmware, etc… [I can’t say enough good things about how well PolyPrinter goes above and beyond to provide top notch support!].

Thanks again for the feedback!

Lisa
:smile:

5

Pronterface’s freezing and Octoprint freezing are different, though, for a few different reasons:

  • Octoprint isn’t running on Windows
  • Octoprint has a different method of checking up on printers (check the Console tab and you’ll see that it sends a few extra Mxx messages while idle)
  • Pronterface was running on what the makers I saw affectionately called “budget laptops that break when a feather touches them”. :stuck_out_tongue:

Pronterface could be debugged by running it through an ‘output’ pipe (“executable.exe >> out.txt”), but that hides the window and doesn’t let you print.

Octoprint debugging could possibly benefit from having a TTY session logged in as root, as every linux flavor I’ve seen redirects hardware errors to root to avoid the user seeing them (if they aren’t labelled critical). However, that’s a really bad idea to do while it’s connected to any network for obvious reasons.

6

This is very concerning. As far as I’m aware, the red button is a hardware reset signal to the printer control board. If the button did not work, then that indicates a hardware issue (loose wire, damaged button / IC, etc.) or a firmware issue (if the button is a software rather than a hardware reset. Someone from Polyprinter would need to chime in here.) If you happen to know about what time this error occurred, I’d like to take a poke through the logs.

Also, being a dev version means there are likely to be more bugs in the software. I chose Far Lefty for this because I’ve heard the most complaints about its behavior. I’m going to try to keep it updated to the very latest dev version to make it more likely that issues will be fixed (and if I find any issues that I can fix, I’ll contribute them back).

My current suspicion is that certain serial commands in a certain order get the printer into an unresponsive state. I think this is what causes the Error: SerialException issue, and why resetting the printer usually fixes that.

The Unauthorized error is a different beast entirely. I’ve seen it give the Unauthorized error on one PC but be fine on another with the same exact file. My suspicion there is a bug in the network whitelist code Octoprint uses to determine which PCs are OK to allow control from. I’ll be looking into this issue more deeply, as it’s something I think I have a better chance of fixing.

7

From what I have gleaned using Octoprint with my Printrbot at home, when you choose CANCEL, or PAUSE, it is as if there are still commands in the “queue” and it will take a few seconds after pausing or canceling before the printer actually responds to the commands. That may be a function of my Printrbot and not necessarily what you are seeing with the printers at DMS, just throwing that out there in case it is pertinent. At first I didn’t think the cancel was working on mine, so I pulled the power on the printer. Later on when I was more patient, I noticed that it would stop after a few seconds.

8

Yes, the printer buffers a few commands, so a pause / cancel will take a few seconds. The remaining issue is that sometimes after a cancel, the printer will stop responding to new serial commands and requires a red button reset to resume operation.

9

I will also go ahead and note that one time up at the space I’ve had one of the printers (Far Lefty again, though this is before the dev OctoPrint) completely restart (as if someone had hit the reset switch for the printer).

  • I was watching the terminal tab for Righty without touching anything (I was printing on Righty and being a geek I wanted to see the exact commands)
  • Far Lefty stops. Completely. Octoprint temperature readings weren’t updating and it was unresponsive to controls (move head left,right, etc… though that’s normal since it was printing)
  • Switched to Far Lefty’s terminal tab, it was spitting out messages as if it was powering up.
  • When it was done, the file Octoprint was reading was marked in red, I couldn’t print any gcode or upload to it
  • Refreshed the Octoprint page, I was warned that the file had failed for “some reason” (that’s literally what the popup said: “The file xxxxx-xxxxx.gcode did not print completely for some reason. Sorry!”).

I have experienced this exactly once so far, and other makers in the room at the time mentioned that they’d never seen Far Lefty do that either.

10

More support to my “weird serial state” theory. Hopefully if it happens again, I’ll have the logs to see what might have caused it. Thanks for the info.