100% accurate bread counting and conveyor control by video camera

Summary

Customer. Baking factory in Europe. The enterprise produces 60 types of bread and bakery products. More than 1000 people work at the plant. Products of the bakery are sold in retail chains in several cities.

Task. To set up accurate bakery counting on the production line and integrate the video counter with the conveyor automation to control it.

Result. Camcontador counts products with 100% accuracy and smoothly controls the conveyor arm.

Situation

A factory has been running product counting with video cameras for several years. The technology helps the operator to count the quantity of products that have passed along the conveyor.

For example, a batch of croissants is moving down the line. The counter’s task is to count a certain amount of croissants, let’s say 40 pieces. Based on the counter data, the operator prepares the products for shipment. If an error is made in counting, the factory will have problems with product accounting, and the stores will find shortages or surpluses of goods when accepting them.

The technology had worked for a long time, but the problem was that the old counter was making errors. When the products on the line came individually, the counting worked fine. But if the croissants were lying perpendicular to each other, or close together, or the packages were not completely cut, the counter missed them. The actual number of croissants that passed through the conveyor belt was more than what was displayed on the monitor.

There are 14 types of products that occur on one line:

– 5 types of croissants (chocolate, condensed milk, strawberry, caramel, sandwich);

– 5 kinds of portioned bread (rye, oat bread, potato bread, buckwheat bread, flax);

– 4 kinds of puffs (apple, cherry, cottage cheese, cheese).

With such a production volume, there were often situations with surplus goods. As a result, the customer started looking for a new way of counting production – a more accurate and modern way.

Task

The customer approached us to solve 2 tasks at the same time:

1) To set up accurate counting of bakery products;

2) Integrate the video counter with the manipulator on the conveyor to control it.

The essence of the integration: the counter counts 40 croissants and moves the manipulator paddle, directing the next items to the next conveyor flow. After counting another 40 items, the video counter moves the paddle back and the cycle repeats again.

In the counter settings, it should be possible to change the number of products that the paddle will separate. Depending on the type of product, different quantities of products are packed into boxes.

Solution

The customer had been using a video counter for several years, so his main question was, “If the current counter makes errors, can your technology count items with 100% accuracy? “.

We were sent several videos to test. The recordings were tricky: murky, blurry, low quality, with a washed out picture, with glued products. These turned out to be the tricky cases in which the old counter was mistaken.

There were bright lights hanging above the line that were illuminating the frame. The customer explained that the extra light was needed for an old video counter. (In contrast, the Camcontador is not light-intensive and can count products almost in the dark. The customer later removed the extra lamps).

Despite the low quality, we trained a counting algorithm on the submitted videos. Our technology correctly counted the products in all the videos. The customer liked the result and we started to implement the counting on the first conveyor line.

The flows were distributed to be captured by different cameras

At first there was one camera above the conveyor, capturing all four flows. From this angle, three lanes were clearly visible. The fourth lane was visible, but part of it was blocked by a wall shared with the third one. In this place there was a blind spot. The camera could not see the products on the 4th flow, which were driving close to the wall, and the counter could not count them.

The solution was simple. The customer hung a second camera to change the angle of the camera. Now the first camera captured lanes 1 and 2, and the second camera captured lanes 3 and 4. In this way all products on all flows became visible.

One camera for all 4 lines:

Two cameras — one on the left:

One on the right:

Upgraded video cameras for smoother pictures

A conveyor line runs 24 hours a day, with different products moving along it throughout the day. Sometimes the speed of the belt speeds up, sometimes it slows down.

The customer’s video cameras captured video at 25 frames per second. At high conveyor speed the frames were blurred and it was difficult to count the products – the camera did not have time to capture the products running along the belt.

The problem was solved by replacing the old video cameras. When the customer installed cameras with a shooting frequency of 30 frames per second, the picture became clearer and the shooting became smoother. Counting production became easier.

We integrated the counter with the conveyor manipulator

In order to link the video counter and the conveyor paddle, a modular discrete output device was connected to the computer. The module was needed to convert the commands from the counter into an understandable signal for the actuator – the paddle.

The mechanism of operation is as follows:

The counter program transmits signals via Modbus TCP protocol → The module converts the signals and sends them to the conveyor via 24 volt output transistor keys → The conveyor paddle slides to the side.

Before connecting the conveyor automation, the customer tested the performance of the mechanism with 24 V lamps. The counter sent signals → The module converted them and transmitted them to the lamps → The lamps lit up. After making sure that everything was in working order, the device was connected to the conveyor.

The logic of the counter operation is as follows:

Count 40 products on line 1 → Transfer the paddle to line 2 → Zero the counter on the operator screen → Count 40 products on line 2 → Transfer the paddle to line 1 → Zero the counter on the operator screen.

In the screenshot below you can see how the paddles distribute the products between the lines. The products move along the belt from bottom to top through inputs 1 and 2 (blue color). The red arrows point to the paddles. Products from input 1 are distributed to lines 1 and 2 (red color). From input 2 – to lines 3 and 4 (red color).

Product selection has been added to the counter settings. The customer needs to count different product quantities for different products. The product type is selected from the menu and the counter understands which quantity of goods to count before the paddle is moved.

The counter stores statistics for any period of time. Data can be downloaded in the form of charts or Excel tables. However, for the customer, this reporting plays a secondary role. The main task of the counter at the enterprise is to help the operator not to make mistakes with the quantity of products on the conveyor. Therefore, at the customer’s request, we improved the counter’s interface, for example, making the numbering on the operator’s screen larger.

Larger fonts on the screen:

Result

After installing the video counter on the first line, the customer was satisfied with the result. The accuracy of the product counting is 100%. The Camcontador counts bakery products correctly and does not make any errors. The integration with the conveyor manipulator works perfectly.

The bakery management decided to scale up the result and install video counters on 6 more production lines. Replacing the old counters with CamContador is cost-effective: our technology counts products more accurately and costs 2-7 times less per camera.

Cost. The customer did not need to buy computers to process the video stream. The computers that worked with the old counter were suitable. The total cost of the project was standard, the customer paid for setting up 2 counting zones at usual rates.

Additional payment was required for integration of the video counter with the conveyor manipulator. Such improvements require a lot of effort: we have to figure out what and how to integrate the counter with, and how much time is required. We can’t disclose the total amount due to NDA.

Duration. The project lasted 2 months. Considerable time was taken up by technical issues: replacement of video cameras, delivery of the discrete output module and its integration with the video counter.

Team. From our side 2 specialists worked on the project. Developer – finalized the program and made integration with the conveyor mechanism. Engineer – trained the neural network to recognize products; for this purpose he collected the largest dataset among our projects – 2425 pictures.

On the customer’s side there was 1 specialist – project manager. He solved all issues lightning fast: from technical to financial.

Egg counting in a poultry farm: 14 production units, 114 cameras and 207 counting zones

Summary

Client. A large producer of chicken eggs. The company’s production facilities occupy 15,000 m². The company employs more than a thousand people.

Task. Implement egg counting with video cameras in 14 production halls.

Result. The CamContador counter is successfully working. The counting accuracy is 99.90-99.93%.

Situation

Birds at the factory are housed in tiered cage batteries. Usually there are 7-8 batteries in the workshops. Each unit collects a certain number of eggs per day.

Monitoring the egg production of the batteries helps to draw the right conclusions to evaluate the performance of the factory. For example:

In order to conduct such experiments, it is necessary to count the number of eggs at each stage of production and for each battery. To do this, a technical solution is needed to measure all the indicators: the production flow entering the plant, the number of eggs produced in the batteries and the output flow.

Often, special counters installed on the equipment are used for such counting. However, according to our data, the CamContador egg counting technology is many times cheaper, and its accuracy matches that of hardware counters. There is no need to buy expensive specialized equipment – you only need video cameras and a computer. The guaranteed accuracy of the egg counter is specified in the contract and is not less than 99.8%.

Having agreed with the above arguments, the factory management chose the CamContador video counter.

The challenge

The client has set the task of implementing egg counting in 13 workshops. Each hall should have between 15 and 18 counting areas. The counting areas include the eggs entering and leaving the production units, as well as the left and right side of the cage batteries where the eggs enter the conveyor belt.

The math is simple. The input stream and the eggs from all the batteries should add up to give the output stream of the shop.

Solution

Usually, customers have no more than 10 video cameras in their workshops, which monitor a certain area. This number is sufficient for counting cans, for counting bread, and for counting poultry carcasses. In the first workshop where we started working, there were 16 counting points at once.

Set up the shooting parameters

The first difficulty we encountered was processing too large a volume of data. At first, the processor did not even have time to process all the video streams. We had to lower the resolution and quality of the footage and simplify the parameters of the neural network so that the computer could cope with the task.

It took us two weeks to experiment with the settings. In the end, we selected the minimum quality level that provides egg counting accuracy above 99.9%.

Fixed the lighting

The CamContador is not light demanding, but in some areas the cameras were shooting darkness. Somewhere in that darkness, eggs were moving on a conveyor belt. This caused errors in the counting.

We asked to add a light source so that we could see the products in the frame. You don’t need powerful spotlights to count products on a conveyor belt. It is enough for a human to see the products. If a human can, then the CamContador can do the same.

Identified and corrected the camera shift

In the first weeks of operation, discrepancies in egg counting appeared. It turned out that one battery had disappeared from the shot. Someone had hit the camera and it was shooting the other side. This caused some of the eggs not to be counted and the math didn’t add up. When the camera was corrected, the counting worked again.

On another similar project, the cameras were shooting products so fast that the counter could not react in time. Eggs were skipping so fast that the program simply didn’t see them. We asked for a different angle and then got a 100% accurate count.

No such problems occurred at this poultry farm. The customer initially complied with our recommendations on equipment and installation.

Javi Martínez, Project Manager

Detected a disconnection

In one of the workshops, we noticed frequent disconnections. They were short, literally a fraction of a second. The counter monitoring constantly sent error messages. Microdisconnects did not affect the accuracy of counting, but it was interesting to understand their cause.

It turned out that there are shops in the factory, from which the computer is half a kilometer or a kilometer away. That is, from the building where the batteries with the bird and video cameras work, the signal goes a kilometer along a fiber-optic cable. Therefore, the connection is sometimes lost.

In other shops with a stable connection, computers stand close, within 100-200 meters, and such disruptions do not occur.

Faced with a typical problem of poultry farms

At all the poultry farms we work with, sometimes there is a problem that cannot be solved. It does not depend on the qualification of specialists, the quality of video footage or the power of the computer. We honestly warn all customers about it.

Eggs cannot be counted correctly if they are riding on the conveyor belt in a slide.

When several eggs are piled on top of each other, even a human can’t realize how many eggs are underneath. The camera is also powerless in this case: it counts what it sees and will not be able to look under the egg.

Fortunately, this problem is rare. Clients realize that there is no getting away from it and it is easier to accept it. In addition, it has almost no effect on the overall accuracy of egg counting.

Results

At the time of writing, the video counter is in operation in 14 workshops. In addition to the egg counting, we have implemented the counting of poultry carcasses in different areas of the slaughterhouse.

The accuracy of the CamContador counter is between 99.90-99.93%.

The customer captures reports on the operation of the system in the form of a table. It shows the input and output flows, the number of eggs per battery, the percentage of accuracy and errors if any.

Due to the low cost and high accuracy of the video counter, the customer took a liking to it. Today, the company has 114 video cameras that capture 207 counting areas.

Our technology allows us to implement a product counter in every production area and see where losses are occurring. This is important to the management of the poultry farm.

In the future, the data collected will be consolidated into one database. The database will make it possible to make summary reports and build your own analytics.

Team. From our side the project is implemented by one specialist. On the customer’s side, one specialist is also involved in the project. He provides remote accesses, submits errors and monitors the operability of the cameras in the workshops.

Cost. 300 € for the first line (camera). 150 € for the each next line (camera). Sum total 1×300 + 113×150 = 17250 €.

Timeline. The launch of the CamContador video counter took one month.

It is gratifying that the company’s management takes the project so seriously. The client buys equipment in full compliance with our recommendations. Famous brands of components and video cameras, modern computers, top video cards, new generation processors, high-speed internet.

Thanks to this, the implementation of the egg counter is fast and the complex system with dozens of cameras works qualitatively and reliably.

Javi Martínez, Project Manager

How we set up bread counting and recognition on the conveyor belt with a video camera

Summary

Customer. Bread manufacturer. The company produces 14 popular types of bread: baguettes, 8 grains, Finnish loaf, rustic bread and others. 

Task. Count and recognize the released products on the conveyor. At the end of the day display a table with the results.

Result. CamContador recognizes and counts all types of bread, including rejects, with an accuracy of 99.8%. A report on the output is built in real time and sent to the accounting department at the end of the shift.

Situation 

A bakery works 24 hours a day without weekends. Every day the enterprise ships 14 kinds of bread to the counters of trade outlets and retail chains. 

With such production volumes, discrepancies in product counting are inevitable. For example, the raw materials were used to produce 1000 units of bread, but the package showed 980 pieces. In order to understand the exact number of products produced, the bakery managers decided to install a video product counter.

Previously, technology did not allow counting products on the conveyor belt with the help of a video camera. Traditional solutions: laser, infrared and pulse counters require additional budget for the purchase of equipment, its installation and customization. Therefore, the customer counted the released products at the packaging stage.

Today, computer vision technology makes it possible to count moving products right on the conveyor belt. It doesn’t matter whether the bread comes out of the oven in rows on pallets or chaotically scattered on the belt. CamContador counts and recognizes products in any form.

Javi Martínez, Project Manager

The challenge

The company has two production lines. The first produces 11 types of bread, the second three. The products on the belt are constantly changing: first one type of bread goes down the conveyor belt, a few minutes later another, then a third.

The customer has set the task to recognize and count all the released products. The counter should determine which type of bread goes down the conveyor belt, count its quantity, and at the end of the day output a table with the results.

Solution

The customer sent videos for each type of bread. At the beginning there were 9, but gradually the number grew to 14 types. We selected 30 screenshots for each product, marked them for counting and trained the algorithm to recognize the products. 

It took 3 days to prepare and train the neural network. When everything was ready, we launched the video counter in test mode and together with the customer started catching errors.

Errors due to the color of the bread

Because of baking conditions, bread sometimes changes color. One batch can turn out pale, while the next batch is ruddy. An algorithm trained on the pale products may make mistakes on the ruddy ones. They have the same shape but different crust colors.

The customer recorded such errors and sent us the video. We re-labeled the frames and re-trained the model. After that, the errors disappeared.

Similar breads in the frame

When the customer sent a video of the 14th type of bread, it appeared to be similar to the other two types, only it was slightly lighter. If there are several units of bread walking down the line, the differences are immediately visible. But when a single piece was in the frame, the algorithm became confused.

Additional training of the neural network would not solve the problem. Therefore, we implemented a refinement at the counting level.

Several minutes pass between different batches of bread. This means that if, for example, 24 products are visible in the frame, they are all the same. There cannot be different breads on the conveyor belt at the same time. If the system found one loaf in a batch that was different from the others, we eliminated this error programmatically.

Thanks to this modification, CamContador has learned a logic close to human logic. The counter has memorized that there cannot be different products in a homogeneous mass of bread. If the algorithm finds several different loaves, the counter will still count them together with the main mass of bread.

A pile of bread on a ribbon

Sometimes the bread goes down the line piled in a heap. The loaves lie on top of each other – one lower, one higher, and a third hides underneath. Such piles of bread have to be removed. The camera can’t see the items underneath.

Fortunately, this problem is quickly solved by installing a bar above the conveyor. This is exactly what the customer did. Now, when the bread rides on top of each other, the crossbar drops the top loaf onto the belt. The result is a single layer of bread that is easy to count and recognize.

Technical difficulties

Capture Quality. Usually, when there are many cameras in a production facility, the CamContador simply counts the products – without recognizing them. For example, this is the case when counting eggs or poultry carcasses. But on this project, the situation was more complicated.

Although there were only two cameras, we had to not only count but also recognize 14 types of products. First, to avoid overloading the network and the processor, we drastically reduced the quality of the video stream. We cut the bitrate and frame rate of the cameras. However, later together with the customer’s representative we decided to improve the picture quality. They raised the bitrate to 4000 kbps and FPS to 20 frames. Thanks to this, images became clearer and counting accuracy increased.

Shooting angle. Initially, one camera was hanging too far away from the production line. This angle made it difficult to recognize the products. There were three types of bread on the tape: white, black and gray. The algorithm made a mistake on the gray bread: the counter assigned it to the black bread and then to the white bread.

The customer lowered the camera so that the loaves could be seen better. We retrained the neural network on the frames from the new angle and restarted the counter. After that, the gray bread recognition errors disappeared and the counting accuracy exceeded 99%.

Problems with Windows 11. Typically, customers use Windows 10 and Ubuntu operating systems. CamContador is fully compatible with these systems. The bakery was the first customer who came with Windows 11. We thought that technically version 11 was not much different from version 10, but it turned out to be different.

There was an unforeseen problem with the drivers for the NVIDIA graphics card and CUDA library. For some unknown reason, the graphics card load was constantly jumping from 30 to 100%. At the moments of peak load it did not have time to process the stream. This caused counting errors.

For a month we changed settings and reinstalled drivers. The problem did not disappear. Then, after consulting with the customer, we decided to switch to the free Ubuntu OS.

The transition was quick and easy. After installing Ubuntu and restarting the counter the video card errors disappeared. For today the program has been running stably and reliably for six months.

The result

Today CamContador successfully recognizes and counts 14 types of bread. Technically, the algorithm even counts 15 types of products, including production rejects.

The cameras are installed on two production lines and count both product flows. The counting accuracy is 99.8%.

At the end of the shift, the customer receives a report on the output by product type. These reports help to control the quantity of bread produced. This information then goes to the customer’s accounting department:

From the first communication with the customer to the launch of the counter on the production line took a month. This is longer than usual projects for counting products on the conveyor belt which are launched in a few days. However in this case we did not just count the same products but recognized 14 different types of bread and defects.

Bread is a rather specific product whose appearance changes due to baking conditions. After the launch, we retrained the neural network 12 times to reduce the error rate, and as a result the counting accuracy was 99.8%. That is, there are only 1-2 errors per thousand units of bread. This is a very high percentage of accuracy compared to conventional counting equipment. However, for the CamContador meter, 99.7-99.9% accuracy is the usual figure that we show in every project and guarantee in the contract.

Javi Martínez, Project Manager

Video example of how the counter works:

Bread Detection, Classification and Counting at the oven exit

The client is a large bakery that produces 13 types of bread. The company has 200 employees.

Task

Make the calculation of bread at the oven exit, make the process cheaper and more accurate.

Result

The client began to use computer vision – artificial intelligence recognizes different types of bread and counts their numbers. The accuracy of calculations has increased to 99.8%, the camera counts around the clock, does not require wages, and does not get tired.

Situation

The client contacted CamContador with a request to automate the counting of bakery products using a video camera. The factory produces several tons of bread per day. Previously, products were counted manually, but the results were unsatisfactory.

The movement of bread along the conveyor is uneven, often the products lie close to each other, so the camera was hung as vertically as possible above the oven exit.

Solution

Stage 1. Recording video in production to train algorithms

The client recorded a video of a conveyor belt along which several types of products are moving.

Stage 2. Data labeling and neural network training

We taught the algorithm to recognize different types of bread on video. On the still frames, each type of bread is manually labelled with a box:

Then the neural network is trained, after which the machine recognizes and counts different types of bread automatically. Algorithms for error-free tracking of the movement of each product unit in the video are a unique development of our engineers.

We had already worked with bakeries before, so part of the project was implemented by analogy with the previous ones. This client’s request was more complex: not just to count, but also to recognize what type of product is currently on the production line, to record in the log not just time and quantity, but also the type of product.

Stage 3. Detection of bread in production

The client installed the equipment:

• an IP-camera with a view of the conveyor belt,
• computer with Nvidia RTX 4070 12 Gb video card (It should be noted that an Nvidia GTX 1660 8 Gb would be enough for such a task.)

CamContador specialists remotely installed recognition and counting software on the client’s computer.

We labeled the data, trained a neural network (model) and uploaded the model to the client’s computer, after which the computer vision technology began to independently recognize and count bread on the conveyor belt.

After several days, it turned out that the machine made mistakes in some cases, so we labeled new images with such cases and re-trained the neural network. After several iterations there are practically no errors left.

Result

Previously, the company had employees working for this task. The cost of the wage fund amounted to 8000 € in year.

The cost of installing a smart counter per line amounted to 1350 €:

— CamContador’s services 900 €.

— Purchase of a video card 450 €.

— The client already had a video camera and a computer, so didn’t have to buy them.

This is a one-time investment – in the future the client will only need to maintain the functionality of the hardware.

The counting accuracy 99.8%.

Savings over the next three years – 24 000 €.

Video egg counter in a poultry farm with 99.9% accuracy

Task

Introduce automatic egg counting with a video camera on three lines.

Result

The eggs were counted on three lines using computer vision with an accuracy of 99.9%. The company began to receive data on the number of eggs produced before packaging in boxes.

Situation

The poultry farm needed to identify the amount of products that did not reach the packaging stage. Previously, production was not counted. To solve this problem, it was necessary to install a video egg counter.

Solution

Stage 1. The client has prepared a video showing the transportation of eggs on the conveyor.

Stage 2. We trained the neural network to recognize eggs. To do this, each egg was manually marked on freeze frames and training was started, so the program began to “recognize” and count the eggs on the video. To solve this problem, algorithms for tracking the movement of each unit of production in the frame were used.

Stage 3. Detection of eggs in a poultry farm

Client:

– installed the cameras on three lines,

– purchased a computer with a Nvidia GTX 1660 8 Gb video card.

Our experts remotely installed a program for product recognition and counting on the customer’s computer. Due to the fact that only one type of product goes through the pipeline, it was enough to train the neural network on one line, there were no recognition errors on other lines, the accuracy was 99.9%.

Launched real-time egg counting on three lines.

Thanks to the solution, we received additional data for analyzing business processes – how many eggs are sent to the next stage of production and what percentage of products do not reach packaging.

Every day the company receives a report and uses the information in its work. In the future, the poultry farm plans to improve the technology and use AI to recognize eggs by shell color.