100% accurate bread counting and conveyor control by video camera

Customer. Baking factory in Europe. The enterprise produces 60 types of bread and bakery products. More than 1000 people work at the plant. Products of the bakery are sold in retail chains in several cities.

Task. To set up accurate bakery counting on the production line and integrate the video counter with the conveyor automation to control it.

Result. Camcontador counts products with 100% accuracy and smoothly controls the conveyor arm.

Situation

A factory has been running product counting with video cameras for several years. The technology helps the operator to count the quantity of products that have passed along the conveyor.

For example, a batch of croissants is moving down the line. The counter’s task is to count a certain amount of croissants, let’s say 40 pieces. Based on the counter data, the operator prepares the products for shipment. If an error is made in counting, the factory will have problems with product accounting, and the stores will find shortages or surpluses of goods when accepting them.

Counting products with a video camera. The monitor shows 2 lines on which the goods are moving and the number of counted products. The video counter used was an outdated technology from another vendor, with analog cameras and additional conveyor lighting.

The technology had worked for a long time, but the problem was that the old counter was making errors. When the products on the line came individually, the counting worked fine. But if the croissants were lying perpendicular to each other, or close together, or the packages were not completely cut, the counter missed them. The actual number of croissants that passed through the conveyor belt was more than what was displayed on the monitor.

There are 14 types of products that occur on one line:

– 5 types of croissants (chocolate, condensed milk, strawberry, caramel, sandwich);

– 5 kinds of portioned bread (rye, oat bread, potato bread, buckwheat bread, flax);

– 4 kinds of puffs (apple, cherry, cottage cheese, cheese).

With such a production volume, there were often situations with surplus goods. As a result, the customer started looking for a new way of counting production – a more accurate and modern way.

Task

The customer approached us to solve 2 tasks at the same time:

1) To set up accurate counting of bakery products;

2) Integrate the video counter with the manipulator on the conveyor to control it.

The essence of the integration: the counter counts 40 croissants and moves the manipulator paddle, directing the next items to the next conveyor flow. After counting another 40 items, the video counter moves the paddle back and the cycle repeats again.

In the counter settings, it should be possible to change the number of products that the paddle will separate. Depending on the type of product, different quantities of products are packed into boxes.

Solution

The customer had been using a video counter for several years, so his main question was, “If the current counter makes errors, can your technology count items with 100% accuracy? “.

We were sent several videos to test. The recordings were tricky: murky, blurry, low quality, with a washed out picture, with glued products. These turned out to be the tricky cases in which the old counter was mistaken.

There were bright lights hanging above the line that were illuminating the frame. The customer explained that the extra light was needed for an old video counter. (In contrast, the Camcontador is not light-intensive and can count products almost in the dark. The customer later removed the extra lamps).

A frame from the test videos sent by the customer. Low quality footage with muddy picture. There are different products along the line. Mostly they are arranged vertically, but sometimes the products are arranged differently. Number 14 on the screenshot is an example of a non-standard arrangement: the product lies diagonally. In such cases, the old counter made a mistake and skipped products

The old counter did not count two packages of products running side by side on the belt

Counting errors also occurred when items followed each other without spacing, as if they were glued together

If the products overturned and went sideways, they were not counted either

When the products were placed close to the wall, the old counter did not distinguish them and missed them

Despite the low quality, we trained a counting algorithm on the submitted videos. Our technology correctly counted the products in all the videos. The customer liked the result and we started to implement the counting on the first conveyor line.

The flows were distributed to be captured by different cameras

At first there was one camera above the conveyor, capturing all four flows. From this angle, three lanes were clearly visible. The fourth lane was visible, but part of it was blocked by a wall shared with the third one. In this place there was a blind spot. The camera could not see the products on the 4th flow, which were driving close to the wall, and the counter could not count them.

The solution was simple. The customer hung a second camera to change the angle of the camera. Now the first camera captured lanes 1 and 2, and the second camera captured lanes 3 and 4. In this way all products on all flows became visible.

One camera for all 4 lines:

The camera captures all lanes at once. Part of lane 4 is hidden by a wall, because of which the products were not visible in the frame

Two cameras

One on the left:

After the additional video camera was installed, the first camera began capturing lanes 1 and 2

One on the right:

The second camera began counting products on lanes 3 and 4

Upgraded video cameras for smoother pictures

A conveyor line runs 24 hours a day, with different products moving along it throughout the day. Sometimes the speed of the belt speeds up, sometimes it slows down.

The customer’s video cameras captured video at 25 frames per second. At high conveyor speed the frames were blurred and it was difficult to count the products – the camera did not have time to capture the products running along the belt.

The problem was solved by replacing the old video cameras. When the customer installed cameras with a shooting frequency of 30 frames per second, the picture became clearer and the shooting became smoother. Counting production became easier.

We integrated the counter with the conveyor manipulator

In order to link the video counter and the conveyor paddle, a modular discrete output device was connected to the computer. The module was needed to convert the commands from the counter into an understandable signal for the actuator – the paddle.

The mechanism of operation is as follows:

The counter program transmits signals via Modbus TCP protocol → The module converts the signals and sends them to the conveyor via 24 volt output transistor keys → The conveyor paddle slides to the side.

Before connecting the conveyor automation, the customer tested the performance of the mechanism with 24 V lamps. The counter sent signals → The module converted them and transmitted them to the lamps → The lamps lit up. After making sure that everything was in working order, the device was connected to the conveyor.

The logic of the counter operation is as follows:

Count 40 products on line 1 → Transfer the paddle to line 2 → Zero the counter on the operator screen → Count 40 products on line 2 → Transfer the paddle to line 1 → Zero the counter on the operator screen.

In the screenshot below you can see how the paddles distribute the products between the lines. The products move along the belt from bottom to top through inputs 1 and 2 (blue color). The red arrows point to the paddles. Products from input 1 are distributed to lines 1 and 2 (red color). From input 2 – to lines 3 and 4 (red color).

The products move from bottom to top and are automatically distributed by the blades.
2 streams at the inlet and 4 streams at the outlet

Product selection has been added to the counter settings. The customer needs to count different product quantities for different products. The product type is selected from the menu and the counter understands which quantity of goods to count before the paddle is moved.

Product selection menu in the video counter settings

The counter stores statistics for any period of time. Data can be downloaded in the form of charts or Excel tables. However, for the customer, this reporting plays a secondary role. The main task of the counter at the enterprise is to help the operator not to make mistakes with the quantity of products on the conveyor. Therefore, at the customer’s request, we improved the counter’s interface, for example, making the numbering on the operator’s screen larger.

Our videocounter uploads reporting for any period of time

Larger fonts on the screen:

We have made the counter font size larger to make it easier for the conveyor operators to see the digits

Result

After installing the video counter on the first line, the customer was satisfied with the result. The accuracy of the product counting is 100%. The Camcontador counts bakery products correctly and does not make any errors. The integration with the conveyor manipulator works perfectly.

The bakery management decided to scale up the result and install video counters on 6 more production lines. Replacing the old counters with CamContador is cost-effective: our technology counts products more accurately and costs 2-7 times less per camera.

Today the counter analyses black and white camera images. No additional lighting or other settings are needed. The design and color of the packaging can be changed – this does not affect the accuracy of counting

Cost. The customer did not need to buy computers to process the video stream. The computers that worked with the old counter were suitable. The total cost of the project was standard, the customer paid for setting up 2 counting zones at usual rates.

Additional payment was required for integration of the video counter with the conveyor manipulator. Such improvements require a lot of effort: we have to figure out what and how to integrate the counter with, and how much time is required. We can’t disclose the total amount due to NDA.

Duration. The project lasted 2 months. Considerable time was taken up by technical issues: replacement of video cameras, delivery of the discrete output module and its integration with the video counter.

Team. From our side 2 specialists worked on the project. Developer – finalized the program and made integration with the conveyor mechanism. Engineer – trained the neural network to recognize products; for this purpose he collected the largest dataset among our projects – 2425 pictures.

On the customer’s side there was 1 specialist – project manager. He solved all issues lightning fast: from technical to financial.