Welcome! Log In Create A New Profile

Advanced

Printing hangs/ freezes

Posted by markbee 
Printing hangs/ freezes
January 08, 2014 08:28AM
Hi,

I tried a more complex print the last days and I'm encountering a sudden print hang with the same print each time after about 6 1/2 hours.

I try to print this:



(it's a prototype end cap for a DIY laser tube).

I tried to print this twice (with and without backwash plate) and both times the print suddenly freezes. There is no serial communication coming back to pronterface. The PSU is running continuously and I can't measure any abnormal temperatures.

It looks like the processor freezes, because I have to reset everything to get the printer working again. As it is quite a large print are there any RAM or other limits which could be the reason for that?

Markus

Edited 2 time(s). Last edit at 01/08/2014 08:30AM by markbee.


XBee & electronics blog: [lookmanowire.blogspot.com]
Re: Printing hangs/ freezes
January 08, 2014 08:43AM
According to the comments in the source code, the firmware is designed to avoid dynamic memory allocation after the startup phase. So RAM should not be an issue. But it does sound rather like a firmware problem. When the firmware has settled down a bit, I may try running our formal verification tools on it to look for problems. In the meantime, can you reproduce the problem in a smaller print, perhaps by slicing the middle section out of your design?



Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].
Re: Printing hangs/ freezes
January 08, 2014 08:53AM
Hi dc42,

you mean only printing the part around where it stops?

It might be pure coincidence that both prints froze after about the same period. Before this print I printed only about 4-5 hours and much smaller/ less complex prints.

Markus


XBee & electronics blog: [lookmanowire.blogspot.com]
Re: Printing hangs/ freezes
January 08, 2014 09:34AM
Yes, it could be coincidence. Did it stop at exactly the same position each time? At what times of day? Have you done any electronics mods to reduce electrical noise and/or increase brownout tolerance?



Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].
Re: Printing hangs/ freezes
January 08, 2014 09:53AM
HI Markus - this might be unrelated, but did you upload the file only once? I've had maybe four prints fail (resulted in a freeze once, the rest doing unprogrammed moves such as rewinding extruder or running off-axis) in the last couple of weeks where I've been uploading similar parts sliced differently, in those four cases reloading the same file from computer to SD card has resulted in a successful run. In the case of the stall, restarting then rerunning resulted in the same stall, but reloading the gcode then running worked fine.

I guess there's no (or inadequate) error checking on uploading to duet /saving to SD card and every now and then a g-code mutates in an inconvenient way. If you still have the SD card and original copies of your file try to do a comparison between them (MS Word or many IDE's should let you do this), which would rule this corruption out (or in),

Ray
Re: Printing hangs/ freezes
January 08, 2014 10:32AM
@dc42

Different times of day, no electronic mods (beside 12V stepdown not working on my board to be replaced) and both prints about the same time of print.

@Ray

I copied the file from the PC to the SD card (which was in the PC) because it was quite big - about 10MB. Uploading would have taken a long time.
I have compared the files and they are identical (checked with UltraEdit filecompare).

/* lamenting on

I think this time I will pass doing further tests with prints taking several hours and debugging this. I'm a bit tired running into minor/ major problems every day or two. I still have a lot of smaller parts to print.
I swear I will only print in .24mm, less than 4 hours, not faster than 20mm/s and will light a candle every morning winking smiley

lamenting off */

Markus


XBee & electronics blog: [lookmanowire.blogspot.com]
Re: Printing hangs/ freezes
January 08, 2014 10:44AM
Quote
markbee
Different times of day, no electronic mods (beside 12V stepdown not working on my board to be replaced) and both prints about the same time of print.

Use of USB power is very marginal due to insufficient dropout margin in the 3.3V regulator, so it's probably best to get RRP to swap your board as soon as possible. Some of us (including myself) experienced print hangs until we made some mods to the electronics, although I can't say there is conclusive proof that these mods are required.



Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].
Re: Printing hangs/ freezes
January 08, 2014 10:48AM
@dc42

It's just that I had a lot of prints now with USB also in the region of about 3-4 hours and _never_ had any hang with the Duet board. But this time the file was much larger, the print time much longer when it freezed.

Markus


XBee & electronics blog: [lookmanowire.blogspot.com]
Re: Printing hangs/ freezes
January 09, 2014 05:51AM
I also had a rock solid printer that suddenly started to hang.
Bobc told me this could be caused by interference on the cable.

After replacing the USB cable my printer seems to be stable again!

Perhaps your problem is related...?


RS-Online Ormerod #263, Kossel mini with Minitronics, Prusa i3 MK2
Re: Printing hangs/ freezes
January 09, 2014 05:56AM
Hi 3D-ES,

thx for the advice. I will check that with some (EMF) stress tests next. The USB cable I'm using is a high quality one, which is also very short. It's directly connected to my laptop.

Markus


XBee & electronics blog: [lookmanowire.blogspot.com]
Re: Printing hangs/ freezes
January 09, 2014 07:55AM
Quote
dc42
Quote
markbee
Different times of day, no electronic mods (beside 12V stepdown not working on my board to be replaced) and both prints about the same time of print.

Use of USB power is very marginal due to insufficient dropout margin in the 3.3V regulator, so it's probably best to get RRP to swap your board as soon as possible. Some of us (including myself) experienced print hangs until we made some mods to the electronics, although I can't say there is conclusive proof that these mods are required.

I haven't had much time to play this week, but I did get one more random lockup around two hours into a three hour print - despite suppression of flyback on both heaters and an "apparently good" 5V rail from the on-board Buck regulator. So that's twice now since cleaning up the flyback issue. This is still a significant improvement over the stock design.

It always seems to happen to me after dark. Two things spring to mind - increased mains noise from dimmer switches and other electrical appliances - and Long-wave radio (skip). Once again this takes me back to my experience with making a metal detector which had a very sensitive front-end for the RX coil. At night, all sorts of gremlins crept in. Maybe related, maybe not.


RS Components Reprap Ormerod No. 481
Re: Printing hangs/ freezes
January 09, 2014 08:22AM
Quote
Radian
I haven't had much time to play this week, but I did get one more random lockup around two hours into a three hour print - despite suppression of flyback on both heaters and an "apparently good" 5V rail from the on-board Buck regulator. So that's twice now since cleaning up the flyback issue. This is still a significant improvement over the stock design.

I'm not sure it's the flyback that's the problem, I think it's the sudden drop in the load on the 12V line and absence of local decoupling causing a spike on the 12V rail. But AFAIR you added a 1uF capacitor across the bed heater mosfet, which alleviates that too. Did you also add a larger capacitor in parallel with C3, to increase the brownout tolerance?



Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].
Re: Printing hangs/ freezes
January 09, 2014 10:15AM
Hi all,

this was quite a short test and quick success.

It was the soldering iron.

The soldering iron is attached to the same socket as my Ormerod is. Switching on an off the soldering iron freezes the print.

I did the print two times and did nothing when it froze the first time. But I can replicate the freeze switching on and off the soldering iron now.

So for the future: I can't work on my workbench, when my 3D-printer is printing?

Markus

Edited 1 time(s). Last edit at 01/09/2014 10:21AM by markbee.


XBee & electronics blog: [lookmanowire.blogspot.com]
Re: Printing hangs/ freezes
January 09, 2014 10:28AM
Hi Markus,
Good detective work. This needs to be pursued further and the noise immunity improved so this does not happen.

We used to do this same ' soldering iron test' (old Weller 24V) in the good old days to check for poor design/construction etc., of equipment and add protection until they would work side by side. Then the equipment may be suitable for release into the hostile electrically noisy world. In the lab UV lamps starting would mess up many things, photocopiers were a potential problem and in one building the lift was a nightmare!

1. Could you try using an extension lead and plugging the printer in another remote socket (without moving the printer or soldering iron).
2. Also moving the soldering iron (but using same mains sockets) would be a good test.

Although not 100% proof, test 1 will help identify whether adding powerline filters will help solve the issue. Test 2. would identify it is a radiated noise and a more difficult task of filtering and suppressing wires to the DUET may be required.

The problem is until we know, a kettle, vacuum cleaner, lightning, radio TX could all be causing us random freezes of printing now and in the future.


Ormerod #007 (shaken but not stirred!)
Re: Printing hangs/ freezes
January 09, 2014 10:45AM
Hi Treth,

I will do those tests tomorrow.

For the time being I added two ferit cores (each at the end of the USB-cable) which I had lying around.
It _looks_ like this helps. I can switch on and off the soldering iron without any freeze afterwards. Even when the cables (USB/ soldering iron) are lying crossed/ above each other.

BTW: It looked like the print hung more likely when I switched the soldering iron OFF.

Markus


XBee & electronics blog: [lookmanowire.blogspot.com]
Re: Printing hangs/ freezes
January 09, 2014 10:47AM
Have you tried using some ferrite sleeves on the power cable on the printer?
Like wrap the 2 core white cable a few times (if long enough) around one of the cores sides and then clamp it shut, that may reduce noise.

Paul


RS Ormerod No 436
Re: Printing hangs/ freezes
January 09, 2014 10:53AM
It sounds like a classic ground loop to me. Your PC is grounded (unless it is a laptop running off batteries), so is the Ormerod, via the ATX power supply. Connecting a USB cable between the two creates a ground loop, meaning that any difference in ground noise on the mains between the two (e.g. caused by turning your soldering iron off) passes through the USB cable. The ferrite cores stop the worst of the high frequency stuff.

Powering the PC and the Ormerod (and preferably nothing else) from the same socket distribution block would probably avoid the problem too.

EDIT: ferrite sleeves on the power wires as suggested by Paul, or even on the mains power cord, should also work. I'm going to add some to the power wires on my build.

Edited 2 time(s). Last edit at 01/09/2014 10:56AM by dc42.



Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].
Re: Printing hangs/ freezes
January 09, 2014 11:14AM
Quote
markbee
Hi Treth,

For the time being I added two ferit cores (each at the end of the USB-cable) which I had lying around.
It _looks_ like this helps. I can switch on and off the soldering iron without any freeze afterwards. Even when the cables (USB/ soldering iron) are lying crossed/ above each other.

BTW: It looked like the print hung more likely when I switched the soldering iron OFF.

Markus
Hi Markus, interesting about the ferites on the USB I posted some time ago that I experienced problems with a laptop + USB +mBed module and ferrite beads were essential, so I will now add some, I use RS309 7940 [uk.rs-online.com] or other similar products. I find the laptop is generally not earthed even with the PSU plugged in (the metal work often floats around 100VAC (at a high impedance) as can be measured from an uncoated screw and a known earth and this acts as an arial injecting noise down the usb lead into the target.
Out of interest I just did a quick test on my DUET and the network connector and USB connector cases do not seem to be connected to 0V's! The SD card case is and the 0V's does connect to the PSU earthed case. But it might be worth investigating the USB and network connector case (screen?) as I'm sure this is wrong.

Just before I started replying I noticed TMD_RS436 suggested ferites also on the mains lead which is a good suggestion from what you are seeing as that is the other route for noise and these are low cost supplies.

Switching off is the worst case situation as this when the switch contacts arc, so again a good observation.

[EDIT] I have highlighted my USB comments in this post and add this following link [forum.allaboutcircuits.com] My comment "I'm sure this is wrong", maybe completely wrong! It would appear that there are alternative views on what is correct/best practice in this area and what is specified, so I withdraw my comment.

Edited 1 time(s). Last edit at 01/09/2014 12:25PM by Treth.


Ormerod #007 (shaken but not stirred!)
Re: Printing hangs/ freezes
January 09, 2014 12:47PM
@Treth

Nope the Ethernet socket shield isn't connected to anything its just soldered to the pads to secure it in, I would of thought it would of been connected to 0V due to it has two transformers inside it and screening it would shield any noise that it may create.

Paul


RS Ormerod No 436
Re: Printing hangs/ freezes
January 10, 2014 07:05AM
Hi all,

I did a lot of tests this morning regarding the CPU freezing. For the tests I homed the x- and y -axis and ran circle.g from SD card. I ran the print (which actually doesn't print) three times for every single test. While doing the print I switched the soldering iron (more or less quickly) on and off. Freezing events are in bold. Where freezing occured at least 1 out of 3 times I did additional tests with ferrite cores.

The results are not considered to be scientific work nor might it be statistical relevant, but those are my observations:

Test 1:
PSU connected with extension lead to different electric circuit. Soldering iron near printer.
Result: No freezing.

Test 2:
PSU connected with extension lead to different electric circuit. Soldering iron about 1 m from printer.
Result: No freezing.

Test 3:
PSU connected to same circuit, soldering iron near printer:
Result: CPU freezes all the time.

Test 3a:
Test 3 with ferrite core on PSU cable near socket.
Result: CPU freezes 1 out of 3 times

Test 3b:
Test 3 with ferrite cores on USB cable (both ends).
Result: No freezing.

Test 4:
PSU connected to same electric circuit, soldering iron about 1m from printer.
Result: CPU freezes all the time

Test 4a:
Test 4 with ferrite core on PSU cable near socket.
Result: CPU freezes 1 out of 3 times

Test 4b:
Test 4 with ferrite cores on USB cable (both ends):
Result:CPU freezes all the time

Test 4c:
Test 4 with ferrite cores on USB cable (both ends) AND ferrit core on PSU cable near socket:
Result:No freezing.

Conclusion: It looks like being on the same electrical circuit without any EMI/ transient/ noise protection might be risky.
I'm not sure about tests 3b and 4b, because I would have expected the opposite (EMI more intensive near the printer/ Duet board/ USB ).
For my part I will do the pragmatic approach and run the printer from a "clean", separate electrical circuit (no transients/ switching noise) and ferrite cores on PSU cable and USB.
I'm still quite a bit surprised that those findings were not observed, respectively revealed while testing the Ormerod for any EMI/ transient sensitivity as I thought those tests are common and mandatory today. As seen above this problem seems to be no rare exception. In another thread one Ormerod user is reporting same trouble while receiving a cell phone message.

[EDITED for typos]

Markus

Edited 3 time(s). Last edit at 01/10/2014 07:52AM by markbee.


XBee & electronics blog: [lookmanowire.blogspot.com]
Re: Printing hangs/ freezes
January 10, 2014 03:55PM
@Markus,

Very interesting tests and with EMI a significant result, it certainly confirms there is an EMI issue and your tests with ferrite cores shows there is a way of reducing the sensitivity. Thanks.

The cell phone effect is probably due to the fact that as the signal strength received drops off the transmitted power from the phone goes up, so in a weak signal area the transmitted power can be quite significant which can have a big effect if on the bench next to the printer. Place one near your scope!

Other things to remember with this issue is someone identified a missing capacitor in the 3V3 switching power supply which may make things even more susceptible than they need be.

The printer bed switching which generates spikes is 'within' the printer environment so any noise spikes here couls still hit the susceptible spot. I did post an idea to get the heater power and switching remote from the DUET supply and board which could improve reliability further. External heater switching.

You may also consider placing a ferrite core on the Vin white power lead at the DUET, I would try just Vin, but you may want to check around both wires. This would at least try to filter at the closest point you can get to the electronics.


Ormerod #007 (shaken but not stirred!)
Re: Printing hangs/ freezes
January 10, 2014 04:11PM
Quote
Treth
You may also consider placing a ferrite core on the Vin white power lead at the DUET, I would try just Vin, but you may want to check around both wires. This would at least try to filter at the closest point you can get to the electronics.

I would definitely put the core around both wires. I think the problem (not the one you get when your phone is close) is ground transients, coupled with the fact that in typical use with the USB cable connected, there will be a ground loop via the host PC and/or other equipment it is connected to. However, I suggest you also put a decoupling capacitor at the Duet board Vin terminals unless you have already modified the board to tame the transient on +12V when the bed heater switches off, because a ferrite core on the power in wires may make the transient worse.

Edited 2 time(s). Last edit at 01/10/2014 04:12PM by dc42.



Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].
Re: Printing hangs/ freezes
April 24, 2014 03:41PM
Hi,

This is an old thread, but I wanted to share my findings.
With 0.57a, I had 3 prints that stopped half way. With one model I could reproduce the problem every time with the first movement. The gcode was
G1 F1800.000 E-1.00000
G1 Z0.360 F7800.000
G1 X78.463 Y77.258 F7800.000

With another model, gcode was this one and it worked fine.
G1 F1800.000 E-1.00000
G1 Z0.240 F7800.000
G1 X50.410 Y50.410 F7800.000

The first gcode travels farthest that the second, and with my bed compensation, there is Z movement too.
Then I found this commit which fix an acceleration issue:
I'm currently using firmware 057y-dc42
I didn't have the problem since.
Re: Printing hangs/ freezes
April 24, 2014 04:46PM
Hmmmm, you could always get a Bunson Burner, and a lump of copper on a stick for a soldering iron... (Sorry that's nearly OFF TOPIC)......lol


Please send me a PM if you have suggestions, or problems with Big Blue 360.
I won't see comments in threads, as I move around to much.
Working Link to Big Blue 360 Complete
Sorry, only registered users may post in this forum.

Click here to login