SEM based vision system

I’ve got a surplus electron microscope that was pretty much free to a good home, it’s on its way which should have working vacuum system which will get me up and running and able to test some parts easily.

I’m trying to understand the requirements for the vision system as I think this might be an area I can help out the most in.
Based on the info on the wiki I found;

“Vision system using 4-sector, independent channel axial Back Scatter electron detector (BSE) Scanning Electron Microscope (SEM) combined with image processing. The pseudo stereo SEM picture data can be converted to true 3D dimensional data (asymmetrical 4-source BSE photometric stereo 3D-imaging)”

“SEM pickup PIN diodes protection cover will be used when printing,”

“SEM ( part assumed to occupy 1/9 of whole print area; 1/9* 300*300=10 000mm2 measurement at every 10µ, …. 4x pictures from 4 pickups give the effect of different angles? “

So basically the electron gun is used in a low power mode? And then scans the work piece. It’s got 4 pickups spaced apart at known angles and there data is linked together to create a 3d image of the work piece. This compared to what it should be will be an error. The error is either removed or taken into account on the next 10 layers or whatever interval is selected.

Some high level intelligence can make things quicker and only scan the area where it should be building up the part. E.g. if creating a 10x10mm part you don’t want it scanning the in tire 300x300mm base

Has anybody looked at what pickups and corresponding hardware is needed? Are there solid state versions of the photo multiplier? Sounds like a nice fpga solution could work with a high-speed analogue per pickup. Similar to a 4 channel fpga oscilloscope hardware setup. 60Msps? I would be keen to get this going. The hardware could then multipurpose with open source scopes.

Cheers,

Pyro

Hi Pyro,

maybe you're interested to get in contact with a friend, that refurbished an old SEM for home use too?

Here's the link to his (German) site: [heupke.com]

---

Viktor
--------
Aufruf zum Projekt "Müll-freie Meere" - [reprap.org] -- Deutsche Facebook-Gruppe - [www.facebook.com]

Call for the project "garbage-free seas" - [reprap.org]

hi
SOunds GReat!
have you read the academic paper? its quite explicit on hardware?
I love contributers to send a mail to me personally every other month or so So I can see if I can offer any support!
kind regards
rapatan
p.s. It might be a help for you if you send me your own email to "rapatan" on this forum as a personal message (privately) on this forum I can send an invite for you to join our MetalicaRap dropbox which has a collection of key documents!

I had a quick read of the paper and it looks pretty in depth and further references more papers that should provide some useful information. I will have another read and double check the hardware they propose.

The hardware will also have to control the x-y coils to get the scan rate in sync with the SEM picture samples

... I've chatted with my friend about refurbishing a SEM for 3D-printing - the worst problem to overcome will be backscattering of vapour and 'secondary ions' ... here you should add some additional EM-'optics' as filters or you'll wreck the SEM

---

Viktor
--------
Aufruf zum Projekt "Müll-freie Meere" - [reprap.org] -- Deutsche Facebook-Gruppe - [www.facebook.com]

Call for the project "garbage-free seas" - [reprap.org]

That''s a very important issue.

I see the wiki says “SEM pickup PIN diodes protection cover will be used when printing,” "

Do you think that a mechanical cover that simple blocks the pickups will work?
Or will the vapor be in the air for awhile, so when the covers open the vapor will get in then even tho its not melting any powder?

Perhaps if these is a repelling magnetic field around the pickup it will deflect the vapor away? I guess the material being printed would have to be ferrous for that to work?

... as you work in a vacuum, the behaviour of the vapour will be different from 'steaming' - maybe you can add some electrostatic blades/capacitors to catch the backscattering particles?

---

Viktor
--------
Aufruf zum Projekt "Müll-freie Meere" - [reprap.org] -- Deutsche Facebook-Gruppe - [www.facebook.com]

Call for the project "garbage-free seas" - [reprap.org]

is there a device in the industry that has to do something similar? There must be a patent or paper describing a method.

If I'm correct a spluttering system coats objects in a vapor? and its this process that we need to control / direct away. So if the spluttering system has a glass bell jar and it looks like you can always see thru the glass. So they must some how keep it contained right?

... search for 'electron beam welding' - [www.isf-aachen.de]

This is used too for adding (cladding) molten metal-tracks on outweared bearings and milling tools for repairing them ...

---

Viktor
--------
Aufruf zum Projekt "Müll-freie Meere" - [reprap.org] -- Deutsche Facebook-Gruppe - [www.facebook.com]

Call for the project "garbage-free seas" - [reprap.org]

SEM, today, imposes great demands on the vision system.
SEM BASED VISION ALGORITHMS are also used for tracking and detection.

I've read the paper here "http://www.ecmjournal.org/journal/smi/pdf/smi98-17.pdf" many times and cant seam to find the sources referenced for the construction of the detectors. they say they use 6 p-i-n diodes. They even have a picture of them in the paper! Has anyone here figured out what type / spec they have? The only pin diodes I find are for Bluetooth circuits etc...

The pin diode is a photodetector from what wiki says. Does this mean that I will need a piece of green phosphorus glass in front of each diode to convert the electron beam to light? or can the pin diode they use detect the back scattered electrons directly?

I can't get to the paper linked above, but I'm assuming the purpose is to detect the x-rays generated by the electron beam coliding with the work material, rather than the low-energy secondary electrons, as some SEM systems do.

I've been searching a bit to find what I could learn about pin diodes. I have found a paper that indicates "Ordinary Silicon PIN photodiodes can serve as detectors for X-ray and gamma ray photons.":

[www.carroll-ramsey.com]

So, it looks like we would have to pick a appropriate PIN photodiode, and a matching scanning energy, so that the x-rays are energetic enough to be detected, but not too energetic that they pass right through the diode.

If you're looking to build a detector, a google search for "pin diode x-ray detector" turns up a plethora of academic papers.

For some pin photodiodes avaliable for purchase, I found this link, that might be a good starting point:
[sales.hamamatsu.com]

Hi jdccdevel,

Thanks for hunting down that information.

Last I read the main page they were looking in the range of a 100kv electron beam.

In your first link there's a statement. "for spectroscopy applications, where the detector diode is optically-coupled to a scintillating crystal such as CsI(Tl) for gamma-ray spectroscopy, or where the detector diode is used directly to detect x-rays below 60 KeV."

That kind of implies that if its at 100kv beam that the detectors will need a scintillator on each sensor?

I see that the company also offers pin diodes with built in crystal's
[sales.hamamatsu.com]

hi
We are now looking between 62KV and 80KV as from others work we think we can get the beam spot size down to 8µ ø some where between 70 and 80 KV but not much less . In soft metals ( aluminium) penetration of electrons does not exceed too badly the 20µ m depth up to 80KV (see link below). We have a chance of running the transformer in air rather than cumbersome oil up to 75kV if we push it with large radius conductor design in power supply secondary side.
So I would consider a 60KV route.
kind regards
MetalicaRap team

PS
electron behaviour in metal simulation ( adjustable accleleration voltage, metal type..etc) no spot size adjustment availible
[www.matter.org.uk]

Hi pyrotronics,

I was able to get to the paper you referenced, and after reading it, it is evident they are detecting the back-scattered electrons directly using the PIN diodes, rather than the x-rays.

A presentation that describes the electron detection process is here:
[stehm.uvic.ca]

Unfortunately, I'm not certain that we will be able to easlily find PIN diodes that will work for this. It appears the electrons must interact with the diode junction itself, and the packaging would prevent this. It also looks like the diodes would be very prone to failure due to metal vapor deposition, which I'm not confident we could prevent 100% reliably, even with a mechanical door.

Regarding the detection of x-rays, I was under the impression that the power supply was going to operate at a different voltage when in SEM mode. Is that not the case? Regardless, the x-ray detection diodes you linked to are rated for x-rays at 120keV, which should be more than good enough. Now we just need to figure out how to use x-rays rather than backscatter electrons for topography. (I've seen some sources using x-rays for crystal topography, but nothing for SEM type systems.)

Jonathan

I have thought about the effects of the metal vapor on the detection elements. I guess there might have to be some protective cover that could get replaced every other build.

Other methods of determining build size could be a thermal imaging camera like this paper describes

link

Are we over looking this? Can we simply use a digital microscope style camera. Have 6-8 of these and build up the 3d image this way?

Or better yet, use a scanning laser style method?

The SEM approach is by far the most elegant and half the parts needed are already there

hi
yes could select a different voltage if needed as could tap lower down on secondary and switch sense feedback voltage divider, but more complex. From a quick calculation from your link I get a radius of 3.5 µm for titanium at 80KV ie resolution of 7µm should be ok?
kind regards
MetalicaRap team

rapatan:

Lower voltage would, I think, be well worth the effort.

I just found an article online about using "off the shelf" CMOS image sensors (Like in a digital camera) for the direct detection of x-rays at 7.49keV.

[proceedings.aip.org]

In the paper, they use a 1024x1280 CMOS detector with a 6.7µm square pixel size, 15µm thick, to directly detect 7.49keV x-rays with efficiency rates of 11.5%.

Using this type of sensor, we should be able to develop a "X-Ray pinhole" type camera, and use two or more of them to do stereographic imaging of the build area. (specificly, we can triangulate the point where the x-rays were generated)

A presentation on x-ray pinhole cameras:
[www.toddsatogata.net]

This setup has many advantages:
- The sensors are commercially mass-produced, and should be reasonably priced.
- We can adjust the focal length of the pinhole camera and distance to the work surface to get the spatial resolution we want. (Using stepper motors & different pinhole sizes if we wish.)
- The sensor can be completely isolated from the build chamber (In the paper they use a mylar window), preventing sensor contamination by metalic vapors.

Modern 1080p CMOS sensors are readily available, This one has a 1920x1080 resolution, and a pixel size of 1.4µm. I can't find any specs on how thick it is though. (Thickness is the primary issue for direct x-ray detection, thicker == better).

[www.ovt.com]

I'm not highly versed in the optics of pinhole cameras. However, my rough calculation suggests that with this sensor, each 1.4µm color pixel corresponds to ~92µm when imaging a 17.8cmx10cm build area, and it can scan at 30fps. Using image post-processing we should be able to locate the point source with sub-pixel accuracy, which will improve the spatial accuracy somewhat. We can also pan the camera around, while focusing on a smaller image area, or use multiple pinholes with shutters.

This is all dependant on detecting lower-energy x-rays (<10keV) We could use the same setup for higher energies, but we would need a "deep depletion" CMOS detector, or we could add a scintillator, both of which would increase the cost significantly:

Example Scintillators:
[sales.hamamatsu.com]

(An interesting thing to note, CsI scintillators can be grown by vacuum deposition. Perhaps, with a working system, we could make our own?)

So does that mean for every photo taken its still one pixel? or is the image meant to capture a full scan of the e-beam?

It sounds like it might be slower at scanning with the 30fps limitation?

pyrotronics:

I was thinking we could probably set up a system where each video frame/picture captured a line. On a flat surface, the line would be straight. Any variations in the surface will cause a deviation in the line that can be used to capture surface topography. There are lots of plans online for using a laser-pointer, a line lens (turns the laser point into a line) and a webcam to do 3d scanning. This would work similarly. Since we would be using custom hardware (i.e. not a webcam) we should be able to drive the sensor hardware harder (more fps), but we would probably start running into dark-current and exposure time issues.

CMOS style image sensors have a "rolling shutter" effect that we would have to account for, and we would have to maintain image synchronization between sensors. That said, (1080/30) = 36 seconds is about the longest I would expect it to take.

We could also decrease the scan time by capturing multiple lines per frame. We can make the assumption that the build area is relatively flat, so as long as the image lines don't cross, we should be OK. Using this trick we should be able to shrink the scan time to 9 seconds (by dividing the scan area into 4x 270 line pieces), assuming we can keep everything bright enough.

This assumes that we only have to raster-scan the build surface on one axis to get the accuracy we want, which may not be the case. If we have to scan in both x and y orientations, and assuming that we can do 4 lines per frame, we're looking at ((1920/4)/30) = 16 seconds for the other orientation. So, a total of 25 seconds per scan. If we can't divide the build area due to exposure-time issues, we would need (36+64) = 100 Seconds per scan layer.

Using a scintillator, so that higher energy x-rays can be captured, the scan can be done in parallel with the build (Assuming 1 min per layer). If we alternate raster directions with each layer, one orientation would need a minimum of 64 seconds, the other 36.

Pyrotronics:

I have found some vacuum compatible PIN diodes with a removable window.

[www.osioptoelectronics.com]

If you download the datasheet, you can see that these diodes are used for electron detection applications, and all the part numbers that start with "XUV" have a removable window, exposing the bare silicon.

As far as I can tell, these should work for the setup discribed in the ecmjournal paper you reference.

I have a question about the detection system that would use these though.

As I understand it, the detector detects changes in the concentration of backscattered electrons in different orientations. This signal changes based on the incident angle between the electron beam and the surface. The information is used to calculate surface normals, which can be used to calcuate surface topography. I can see how this gives good surface detail, and it's nice because the electron beam can be used directly to adjust the image resolution. We can use the information to make sure each slice has the appropriate outline (x and y), and to make sure the build surface is flat.

However, how accurate will this system be in detecting the absolute height of the build surface? From what I can tell it would be very accurate in the X and Y orientations, and give good relitive depth information (+ and - in the Z orientation) but it doesn't seem to me it would give any absolute Z value to base the relative Z information on.

It seems to me that this could be a big issue with this sensor setup for 3d printing. Do we need a secondary sensor for detecting absolute Z height? Or am I overstating the importance of this?

Maybe we need a combination of both types of sensor? The pinhole camera gives good absolute x,y, and z) position information, whereas the electron detector can resolve better surface detail.

... which detail resolution do you need for 3D-scanning the surface?

I'm on structured-light scanning with modified DLP-beamers and "David-laserscanner" and made some tests with replacing the light source with LED's or diode-lasers.

Here some infos in English ... and here (post #8) some more infos/images to the used diode-lasers in German ...

With an unmodified dental-scanner with laserl-ine I've got resolutions of 10 microns ... with reducing the projected beamer-image with a refocussing lens after the output the effective resolution should be in the single Microns range.

---

Viktor
--------
Aufruf zum Projekt "Müll-freie Meere" - [reprap.org] -- Deutsche Facebook-Gruppe - [www.facebook.com]

Call for the project "garbage-free seas" - [reprap.org]

VDX:

10 Microns is really good for a structured light scanner. Can you provide a link to the specs for the dental-scanner you use? I don't see that info in the threads you reference above.

What sort of scan area can you do at that resolution, and how long does it take to capture a scan?

Since the printed surface is freshly melted metal, and the inside of the build chamber is polished metal, wouldn't optical reflections be an issue for this type of scan?

How well would it work for detecting the difference between melted and unmelted powder? The scanner will only be able to see the top layer of the build, as the rest of the object is obscured by the unmelted metal powder. Each image has to very accurately represent the surface topography, ideally it has to be able to differentiate between individual metal powder grains (25 to 45 microns, according to the wiki) and a fully melted metal surface.

Since you have a working scanner, perhaps you could do some testing?

Icing (confectionary) sugar has a particle size of 15 to 25 microns: [www.biscuitexpert.com] Perhaps you could take some test scans of some icing sugar on an aluminum plate? If your scanner can detect the individual grains, I would say it would probably be accurate enough.

Judging by the pictures on the threads you've referenced, I assume that this system would not be able to image the entire build area at once at a 10 micron resolution. Would there be any problem mounting the entire optical assembly on a X-Y carriage in the vacuum chamber? (heat, etc)? If it can't be mounted in the chamber, we need to develop a optical system that can accomidate a stationary projector.

... that's a bunch af questions at once - but I'll try to answer ...

- the dental-scanner is an old i3Dscan from imes-icore:

- the scanning range is maybe 20x20mm, the two scanning BW-cameras have 1200x1024 pixels @30fps and the line-laser throws a really fine laserlinie over the center of the turning table

- here a point-cloud of a mini-figure with a 5mm big head (one of the first scans with medium accuracy, before I switched to SL-scanning):

- the scanning time per job is some seconds to maybe 30 seconds for a full 360°-scan, depending on the rotation speed and defined resolution ... faster with only partial scans.

- the possible resolution is visible when calibrating - here it shows the difference to the last calibration with max. 8 to 10 microns for different angles, positions and calibrating points.

With David-laserscanner structured light scanning is different from scanning with laserline - here the accuracy for normal situations is maybe 5 times better with SL than with laserline when using same setup and camera settings.

With the laser I have problems to get lines smaller than 50 microns, but with a reducing lens before the beamer optics or an optically modified DMD-setup and comparable image sizes of e.g. 20x15mm I'll receive a pattern with a resolution of better than 20microns spotsizes or linewidths.

Glossy parts or specific colours are really problematic for scanning (so I'm covering them with matte-spray), but when sintering from powder, the surface should be diffuse/sandy enough to hold the contrast ...

I'll try to scan some micro-parts with the unmodified dental scanner, but it could last longer, until I can configure a SL-setup for this dimensions too ...

---

Viktor
--------
Aufruf zum Projekt "Müll-freie Meere" - [reprap.org] -- Deutsche Facebook-Gruppe - [www.facebook.com]

Call for the project "garbage-free seas" - [reprap.org]

Forgive my ignorance but will the un melted powder be held together strong enough to support a piece being printed on top? Or will it require a support structure?

It just occurred to me, can we fuse the actual object being printed. but partially melt the support structure and bead blast it off later?

I've always assumed there will be a support structure, and it will be fused to the base.

... if the fused parts won't warp, the surrounding/underlying powder would be enough - otherwise you'll need some sort of breakaway-support ... best would be a grid of thin vertical needles or 'trees' with branches and thin connection points to the bottom surface of the supported object.

For hollow or 'stacked' parts you'll need this support inside the hollow regions of the object too ...

---

Viktor
--------
Aufruf zum Projekt "Müll-freie Meere" - [reprap.org] -- Deutsche Facebook-Gruppe - [www.facebook.com]

Call for the project "garbage-free seas" - [reprap.org]

I remember reading about this on the wiki (The manufacturing walkthrough touches on this, but I can't find the exact quote I was looking for.)

The powder is pre-heated to 20 deg C below melting to lightly sinter it in place. This provides the support structure and maintains the shape when the material is melted. This also reduces thermal stress in the finished part.

It is my understanding that finished parts may require some polishing to remove the remnants of the scintered powder. Polishing may also be required to improve finish in areas in the electron beam shadow. (This would, I assume, be dependant on the powder particle size.)

Incidentally, this is why having 10 micron or better resolution is important in our vision system if possible. At that resolution we can:
1) Measure the particle size for the material we are working with. (Usefull for estimating powder temperature and required beam energy.)
2) Detect the transition between discrete particles and melted metal (allows confirmation of build dimensions, calibration of beam energies, etc)

I have not seen any discussion of hollow parts, and that has been a question in my mind for a while. In an attempt to keep this thread on-topic, I'll start a new thread for that discussion.