Optimizing Injection Molding Parameters Using Reinforcement Learning with Prior Knowledge

Injection molding is one of the most widely used polymer processing techniques and is responsible for producing a significant portion of plastic products ^[1]. However, the process involves a complex interplay of technological parameters that must be carefully examined before starting mass production ^[2].

Incorrect parameter settings can lead to numerous production defects, such as warpage, flow marks, short shots, and sink marks ^[3]. Additionally, different injection molding machines may have varying control mechanisms, leading to differences in optimal machine settings ^[4].

Polymers themselves also vary greatly in their thermal properties, necessitating different processing parameters for different materials ^[5]. Even slight changes in injection molding settings can affect product quality ^[6]. Moreover, the mold’s design and material play a crucial role in the overall process ^[7].

For instance, the gate design can significantly influence process optimization ^[8]. Determining the ideal machine settings is a challenging task—if the cavity is filled too quickly, it may cause flash; if it’s filled too slowly, a short shot may occur ^[9].

These challenges stem from the versatility of injection molding, which accommodates a broad range of materials and parameters.

Each material exhibits unique mechanical and thermal characteristics based on its molecular structure ^[10]. To fully leverage a polymer’s potential, its specific properties must be considered when designing a mold for a new product ^[11].

Given the complexity and multi-variable nature of injection molding, analyzing the effects of various parameters and their interactions is essential.

Traditional mathematical methods offer one approach. For example, Mukras et al. ^[12] used a central composite design involving seven injection molding parameters to minimize shrinkage and warpage.

Roy and Li ^[13] developed an information model based in part on polymer properties to reduce waste during production. Chen et al. ^[14] demonstrated how sequential design techniques could optimize injection molding processes, while Barghash and Alkaabneh ^[15] built regression models to describe the relationships between quality metrics and processing parameters.

Another approach is to analyze parameter effects using simulations instead of physical trials. Modern computational tools, particularly finite element–based software, enable detailed simulations of the injection molding process ^[16].

These simulations are especially helpful in visualizing how materials fill molds during the process ^[17]. Simulation-based experimental design methods, such as Taguchi analysis, can be developed with these tools ^[18].

They are also useful when dealing with mold materials that exhibit non-linear elastic behavior ^[19]. However, it’s important to recognize that different software may apply different boundary conditions, leading to varied results ^[20]. Although simulation outcomes may not always match actual injection molding results, they still provide valuable insight into the influence of process parameters, which can be applied in real-world scenarios ^[21].

In recent years, machine learning–based approaches have gained popularity as alternatives to traditional methods in injection molding ^[22]. For instance, Gim and Rhee ^[23] applied neural networks to model the relationship between in-mold pressure and part weight.

Gao et al. ^[24] demonstrated the flexibility of machine learning by using it to design conformal cooling channels in injection molds, achieving more uniform temperature distribution during molding.

Moreover, numerous studies have explored the use of machine learning for predicting product quality ^[25], ^[26], ^[27].

The emergence of Industry 4.0 in the early 2010s introduced a new industrial paradigm. Although it lacks a strict definition, Industry 4.0 is commonly understood as the fourth industrial revolution—marked by interconnected machines, sensor-driven manufacturing, and digital communication ^[28]. Injection molding has also embraced this concept.

For example, Khosravani et al. ^[29] described Industry 4.0 as a fusion of injection molding with technologies such as 3D printing, virtual reality, and generative design. Their intelligent system enhanced production efficiency and reduced costs.

Rousopoulou et al. ^[30] developed a control system that combines cognitive analytics with machine learning for anomaly detection and self-retraining capabilities.

Similarly, Farahani et al. ^[31] defined Industry 4.0 as the processing and interpretation of sensor data, using feature extraction to detect faults during injection molding.

Benešová and Tupa ^[32] predicted that the fourth industrial revolution would transform job roles in manufacturing, much like previous industrial revolutions.

Combemale et al. ^[33] anticipated that low- and medium-skilled tasks would increasingly be handled by machines. In the context of injection molding, this transition may necessitate the development of intelligent systems capable of configuring machines autonomously.

Among machine learning techniques, reinforcement learning (RL) stands out as it involves learning through interaction with the environment to accomplish tasks ^[34,35].

This characteristic makes RL a suitable candidate for control systems aligned with Industry 4.0 principles. Some studies have already demonstrated its application in injection molding—for optimizing processing parameters ^[36], managing production schedules ^[37], and adjusting settings for defective products ^[38].

A key question arises: how do reinforcement learning–based control systems compare to traditional ones? According to Ugurlu et al. ^[39], RL-based systems are more adaptable to new environments, while conventional systems tend to offer greater stability.

For example, PID controllers are effective in linear, predictable environments but less suitable for environments subject to sudden changes ^[40]. On the other hand, a major limitation of RL-based controllers is their demand for extensive training data and time ^[41].

Traditionally, setting up an injection molding machine for a new product relies heavily on the technician’s experience and a trial-and-error approach.

However, in line with the Industry 4.0 vision, these tasks are expected to be increasingly automated. This study introduces a novel method for configuring injection molding machines throughout various phases of the molding cycle.

The approach is based on reinforcement learning—specifically, the actor-critic algorithm—and leverages prior learning from earlier products to optimize settings for new ones.

The proposed method is demonstrated for optimizing the filling and holding phases, accommodating variations in material type, gate design, part thickness, geometry, and size.

Materials and Methods

Injection Molding and Experimental Setup

The experiments were conducted using several injection molds (see Fig. 1) and various Arburg injection molding machines: the Allrounder Advance 270 S 400–170, the Allrounder 320C 400–170, the Allrounder 420C 1000–290, and the Allrounder 470 A 1000–290, all from Arburg GmbH + Co., based in Loßburg, Germany.

Two types of polymer materials were used: Terluran GP-35 acrylonitrile butadiene styrene (ABS) and Ingeo Biopolymer 3100 HP polylactic acid (PLA).

While most parts were produced using ABS, both ABS and PLA were used for the lid components (Fig. 1b) and the fan-gated plates (Fig. 1e).

Two experimental series were conducted. The first focused on optimizing the filling phase, using plate-like components and the lid product (Fig. 1b). The second set of experiments investigated the holding phase, using the small lid, the lid, and a 1 mm thick plate (Fig. 1a, b, d).

a) small lid with 16 cavities b) lid product c) plate with a film gate (2.5 mm) d) plate with a film gate (1 mm) e) plate with a fan gate (1.2 mm) f) plate with a double gate (1.2 mm).

> Injection-Molded Parts for Filling Phase Analysis

To evaluate the effectiveness of utilizing prior knowledge from different products, a series of injection-molded parts was produced using variations in gate design and wall thickness while maintaining a consistent base geometry of 80 mm × 80 mm. These variations included three distinct gate types and three different thicknesses.

Additionally, to assess material-specific behavior, plates were molded from PLA using a fan gate, enabling comparison against ABS. A lid component with a completely different geometry was also included to examine how well the method generalizes to structurally dissimilar parts.

The objective of this investigation was to train a reinforcement learning algorithm to identify optimal injection molding settings for achieving complete cavity filling. The algorithm was expected to apply learned knowledge from one product type to optimize settings for another.

The experimental setup included 1 mm and 2.5 mm thick plates using a film gate, along with 1.2 mm thick plates molded with fan and double-edge gates.

During the filling phase experiments, in-mold pressure data was collected using two different sensor systems: the Kistler CoMo system (Kistler Group, Winterthur, Switzerland) and Cavity Eye in-mold sensors (Cavity Eye Ltd., Kecskemét, Hungary).

After each molding cycle, a Microsoft LifeCam Cinema webcam (Microsoft Corporation, Redmond, WA, USA) was used to capture images of the parts post-mold opening.

Two key process parameters were varied in this study: injection rate and switch-over volume. These variables were systematically altered to produce a range of outcomes, including short shots (incomplete filling), overfilled parts (excessive in-mold pressure), and properly filled parts.

Switch-over volumes were set at 22, 20, 18, 16, 14, 12, 10, 9, 8.5, 8, 7.5, 7, 6.5, and 6 cm³. Each switch-over setting was tested with four different injection rates: 15, 30, 45, and 60 cm³/s (see Table 1 for complete parameter settings).

For the 1.2 mm thick plate, five samples were produced for each parameter combination. In the case of the 1 mm plate, three samples were made per combination. Repeated trials confirmed that the standard deviation across different setting combinations was statistically insignificant.

As a result, only a single setting combination was used for molding the 2.5 mm thick plates, and its variance was applied across the remaining settings for consistency.

For the lid product and the PLA-molded plate, three samples were created for each switch-over volume at a given injection flow rate. These datasets were used to analyze and predict machine-induced variability.

All collected data from these experiments were then utilized to simulate the learning behavior of the algorithm under different molding conditions.

Table 1. The injection molding settings for the filling experiments.

> Injection Molded Parts for Holding Phase Analysis

To assess how prior knowledge influences the algorithm’s effectiveness in optimizing the holding phase, products with different sizes, geometries, and materials were injection molded.

Three types of products were selected for the study: a lid with a complex geometry, a 1 mm thick plate, and a small lid with a simpler design but produced using a 16-cavity mold (see Fig. 1a–d).

Test specimens included both ABS and PLA versions of the complex lid product to evaluate the impact of material differences. The two key process parameters varied during the experiments were holding pressure and holding time, as these primarily define the holding phase.

The resulting part weights were measured using an Ohaus Explorer analytical balance (OHAUS Europe GmbH, Uster, Switzerland).

Holding pressure was generally varied from 0 bar to 1000 bar in increments of 100 bar. However, two exceptions were made:

For the 1 mm thick plate, flash occurred at pressures above 800 bar, limiting the upper pressure bound.

For the small lid, due to the complexity of weighing parts from a 16-cavity mold, only four pressure levels were tested: 0, 200, 600, and 1000 bar.

Holding time was adjusted in 0.5-second intervals, ranging from 0 to 5 seconds. Again, an exception was made for the small lid, since its weight showed negligible variation beyond a 3-second holding time.

For the plate, which used a film gate that froze more slowly, the holding time was extended up to 6 seconds.

A complete list of the additional molding parameters is provided in Table 2. For each setting combination, five samples were produced for both the ABS lid and the small lid. Similar to the filling phase analysis, the data variance did not show significant differences across the different settings.

Consequently, for the plate, only three samples were produced at each holding time with a fixed pressure of 400 bar. These results were used to estimate variance across the parameter space.

For the PLA lid, three samples were produced at each holding time across three pressure levels: 0, 500, and 1000 bar. These datasets were used to train and validate the reinforcement learning algorithm’s capacity to generalize from prior knowledge and adapt to new configurations during the holding phase.

Table 2. The injection molding settings for the holding experiments.

Digital Image Processing

In the filling phase optimization process, the learning algorithm relies on both in-mold pressure data and visual information captured from the molded product.

To enable this, the product image must be processed to determine the degree to which the cavity was filled. This was accomplished using Matlab R2023b (MathWorks Inc., Natick, Massachusetts, USA).

The image analysis was conducted through a structured detection algorithm, designed to identify the molded part within the captured image. The image processing pipeline involved several key preprocessing steps:

• 1.Conversion to a binary image
• 2.Recognition of the sprue
• 3.Gate detection
• 4.Noise filtering

These steps ensured that the product could be accurately identified and analyzed for completeness. The full process of the implemented image detection algorithm is summarized in Table 3, and a visual representation of the detection workflow is shown in Figure 2.

(from left to right) transformation to binary image, sprue recognition, gate detection, noise filtering.

Table 3. The pseudocode of the image processing algorithm.

During the binarization step, most non-product elements—such as mold geometry, fastening screws, and other background artifacts—were effectively removed from the image.

However, some features, like ejector pins and flash, could not be eliminated through this process alone. Since only the upper molded product was relevant for analysis, the images were initially divided into upper and lower halves.

The sprue, located between the two product halves, served as a key reference point for identifying the upper part.

To detect this, the image processing algorithm scanned the binarized image row by row, starting from the top-left corner. For each row, it recorded the position of the first white pixel, corresponding to the left edge of the sample.

This sequence of positions formed a function that could be plotted against the image’s vertical pixel index (rows), as illustrated in Figure 3.

Around the center of this function, a local extremum was consistently observed, indicating the precise location of the sprue. This information enabled the algorithm to isolate and analyze the appropriate product region for further processing.

Once the location of the sprue is identified in the image, only the region above the sprue is relevant, as this contains the specific product under analysis. The next step in the process is gate detection.

To accomplish this, the algorithm leveraged the fact that the profile of the first white pixels (i.e., the product’s left edge in the binarized image) remains relatively consistent along the product’s length.

Based on this, the algorithm computed the derivative of this profile function. A significant change in the derivative—crossing a predefined threshold value—was interpreted as the transition point from the gate to the main body of the part.

This approach allowed for accurate identification of the gate’s end and the product’s beginning, as illustrated in Figure 4.

Once the gate location is identified, only the image region above the gate is relevant, as this contains the actual molded part.

Within this selected area, the algorithm examines the connectivity of white pixels to determine which regions correspond to the product.

While many white pixels appear in this section, not all of them are part of the molded component. Some represent ejector pins or light reflections (Fig. 5a).

These small, disconnected white regions were identified and filtered out to avoid misclassification.

In cases involving a double gate design, the molded product may appear as two separate segments when the switch-over volume is high, meaning less material was injected and the cavity wasn’t fully filled.

The algorithm accounted for this by recognizing and treating the two lower areas in the image as a single product entity (Fig. 5b).

The remaining white pixels in the image represent the molded product, making their quantity a reliable indicator of cavity filling.

To validate this, product weights were measured for a specific injection flow setting corresponding to each product geometry and then compared with the number of white pixels identified by the algorithm. The resulting relationships (Fig. 6) showed a strong positive linear correlation at a significance level of 0.05.

However, as the switch-over volume decreases, the number of detected pixels exhibits noticeable saturation.

This effect is less apparent in the weight measurements—except for the lid product—because during injection molding, continued material injection after cavity filling compresses the melt cushion inside the mold, increasing part weight.

Nonetheless, this additional mass accumulation is undesirable as it can introduce residual stress in the product or cause the mold to open under high in-mold pressure, resulting in flash defects.

Therefore, fine-tuning of the product weight is better addressed during the holding phase optimization, since control over that phase is more effective for managing final part weight.

In-Mold Pressure Processing

For optimizing the filling phase, in-mold pressure sensors were utilized to detect both overfilling and the cavity filling time.

Although the injection molds are equipped with multiple in-mold sensors, their positions vary depending on the product design. Therefore, for each product, two sensors were selected for measurement: one positioned closest to the gate and the other near the center of the part.

Specifically, sensors CH2 and CH4 were used for the 1.2 mm thick plates (Fig. 7a), sensors F1K and F2K for the 1 mm and 2.5 mm thick plates (Fig. 7b), and sensors C1SG and C1S2 for the lid product (Fig. 7c).

To detect overfilling, the maximum pressure recorded by the middle sensors (CH2, F2K, and C1S2 as shown in Fig. 7) was used. Ideally, sensors located at the end of the cavity would be preferred, but since the Kistler sensors did not include end-of-cavity positions, the middle sensors were chosen for measurement and analysis.

Both the gate sensors (CH4, F1K, and C1SG) and the middle sensors (CH2, F2K, and C1S2) were employed to calculate the filling time (Fig. 8). For each sensor, the filling time was determined by identifying the moment when the pressure first reached or exceeded 5 bar. The filling time, denoted as tfill, is defined as the difference between the times recorded by the two sensors.

A detailed discussion of the state definitions derived from image processing and pressure curve evaluation is provided in Section 2.4.1.

Learning algorithm

The goal of this study was to demonstrate how prior knowledge from one injection mold can be applied to another using a learning algorithm capable of deciding which settings to adjust and by how much. To achieve this, an actor-critic algorithm was developed, as outlined in Table 4. The algorithm’s actions consist of modifying injection molding parameters.

For filling phase optimization, the parameters adjusted were the switch-over volume and the injection flow. During holding phase optimization, the varied settings were holding pressure and holding time.

In the filling phase, the state was defined using the detected product size from the image, the maximum measured pressure, and the measured filling time. These were normalized relative to ideal or target values. For the holding phase, the state represented the product weight as a percentage of the target weight.

The algorithm employs state aggregation, meaning it groups similar states and evaluates them collectively. For example, products weighing 15 g and 16 g might be considered equally good, whereas a product weighing 20 g would have a different evaluation. The quality of a state is computed using the state–value function:

The quality of a given state is determined by the state–value function (s,w), where x₈ represents an index vector indicating the specific aggregated state the algorithm is currently in, and w denotes the weight vector associated with those aggregated states.

This same state aggregation approach is utilized in the policy function approximation π(α∣s,θ), which evaluates the effectiveness of each potential action within a given aggregated state.

Within the policy function, x_h acts as an index vector identifying which action the algorithm selects in a particular state, while θ is the corresponding weight vector for each state–action pair. At the outset, both w and θ are initialized with values of 0.5 for all discrete states and their respective actions.

As the learning process progresses, the algorithm continuously updates the w and θ vectors to refine its estimation of each state’s value and the value of each state–action combination. These ongoing updates enable the algorithm to make increasingly optimal decisions, gradually aligning its outputs more closely with the desired target values.

Table 4. The pseudocode of the presented learning algorithm.

The algorithm also includes a restart option. This feature allows the system to reset to the initial injection molding settings after a specified number of learning steps (t_step) or at a predetermined time (n_restart). This functionality enables the analysis of whether repeating the initial injection molding settings during the learning process improves the algorithm’s ability to adapt to a new product.

During training, the algorithm was taught using data obtained from real injection molding experiments. When the algorithm attempted to apply settings that fell between the measured data points, interpolation was used to guide the learning process.

This method allows for a practical workflow: by conducting a design of experiments, the algorithm can be pre-trained offline with existing data and then fine-tuned online during actual production. This combined approach, illustrated in Fig. 9, leverages data processing and machine-to-machine communication—key elements of the Industry 4.0 paradigm.

The ultimate goal is to integrate the algorithm directly with the injection molding machine to enable real-time production setup. However, this study utilized pre-measured offline data to simplify and accelerate the analysis.

In this setup, the algorithm determines how it would adjust the machine settings, and the corresponding output values for those settings are retrieved from the database. In practice, it’s impossible for the injection molding machine to produce two consecutive products of exactly the same quality, due to variations in the melt and limitations in machine control precision.

To account for this variability, the algorithm adds normally distributed random noise (mean zero, variance based on measured standard deviation) to the database values whenever they are accessed. This approach models the inherent randomness in the injection molding process during the experiments.

For filling phase optimization, the learning algorithm adjusts two parameters: the injection flow and the switch-over volume. A smaller switch-over volume means more melt is injected into the cavity.

Once the cavity is fully filled, further reducing the switch-over volume causes the melt to be compressed inside, increasing the in-mold pressure. The injection flow controls the speed at which the melt enters the cavity.

Changes in flow speed affect shear stress, viscosity, temperature, and specific volume of the melt. Naturally, higher injection flow leads to faster cavity filling. The algorithm learns how to vary these settings to optimize filling under different conditions.

For holding phase optimization, the algorithm modifies the holding pressure and holding time. These parameters govern how much extra melt is injected after the cavity is filled. As the melt cools, it shrinks and may not completely fill the cavity, so the additional melt compensates for this shrinkage and slightly increases the product’s mass.

Higher holding pressure pushes more melt into the cavity, but only for the duration defined by the holding time. Therefore, extending the holding time also increases the amount of melt injected. This effect lasts only until the gate between the cavity and the channel solidifies (freezes).

By adjusting holding pressure and time, the algorithm can fine-tune the product’s final weight.

At the core of the learning algorithm is the reward function, which provides feedback from the environment. The algorithm uses the reward to evaluate whether its decisions are good or bad. The reward acts as a penalty because the algorithm aims to reach 100% (within a set tolerance), and any deviation from this target results in a negative reward. The reward function is defined as follows:

Here, R represents the reward function, and S_actual corresponds to the actual state value—either for filling (S_fill) or holding (S_hold). As indicated in Equation (1), a value of 100 is considered the target or ideal state, with any deviation resulting in a penalty reflected as a negative difference. The use of the negative sign is essential to ensure proper convergence during the learning process.

The algorithm’s performance can be evaluated either by the actual state achieved at each step (Fig. 10a) or by the corresponding cumulative reward (Fig. 10b). Both figures illustrate the median and interquartile range, and they convey equivalent information, as they can be converted using Equation (1).

However, for the filling phase optimization, the tolerance limits are asymmetrically distributed around the target state (−3% and +2%), making it more effective to represent results in terms of state values.

On the other hand, for holding phase optimization, the tolerance boundaries are symmetrically placed around the target (±0.5%), making cumulative rewards the preferred representation—consistent with common practices in academic literature.

> State Space for the Filling Phase

The state value used for optimizing the filling process is derived from three components. The first component is based on the detected product within the captured image (Equation 2). In this context, n_pixed represents the number of white pixels identified during the part detection process, S_₁ denotes the image-based state, and n_goal corresponds to the expected number of pixels representing a fully filled product in the image.

The second component of the state value is derived from the filling time. While shorter filling times may benefit the learning algorithm, they do not always equate to better outcomes. To address this, a ramp function (Equation 3) is applied to transform the data accordingly. In this equation, t_fill represents the actual filling time (as illustrated in Fig. 8), S_₂ denotes the time-based state, and t_goal is the target or optimal filling time. This function converts the ratio into a percentage, reflecting how closely the actual filling time aligns with the ideal target.

The third component of the state was determined using a similar approach. In this case, a predefined pressure limit was applied. If the measured maximum pressure (Pₘₐₓ) remained below this threshold, the value was set to 100. However, if it exceeded the limit, the ratio between the measured and limit pressures was used instead (Equation 4). In this equation, P_ₘₐₓ represents the recorded peak pressure, S_₃ denotes the pressure-related state, and P_limit is the designated maximum allowable pressure.

The overall filling state (S_fill) is defined by combining the three individual components using a piecewise function (Equation 5). This function determines which state values to consider based on the pixel ratio from the image. If the pixel ratio is below 97%, the state is derived solely from the image data.

When the pixel ratio falls between 97% and 99.5%, the function calculates the power mean of the image-based ratio and the filling time ratio. Once the pixel ratio exceeds 99.5%—indicating that the cavity is almost completely filled—the power mean of the pressure ratio and the filling time ratio is used instead.

This structure ensures that if the product is underfilled, pressure data is excluded from the evaluation, as the filling target has not been met. Conversely, when the filling is sufficient, both pressure and fill time contribute to assessing the quality of the process.

Based on Equation (5), the state within the test space for each product was determined using the measured data (see Fig. 11). A consistent filling time target (t_goal) of 0.2 seconds and a pressure limit (P_limit) of 220 bar were applied across all products, with two exceptions: the 2.5 mm thick plate and the lid product.

For the 2.5 mm plate, adjustments were necessary because thicker products do not reach the same pressure levels as thinner ones, and filling a larger volume of melt inherently requires more time. As a result, the pressure limit for this plate was set to 50 bar, and the filling time target was extended to 0.4 seconds.

In the case of the lid product, which features a smaller gate and a more intricate design—including tubes, ribs, and areas of varying thickness—the filling time target was adjusted to 0.3 seconds, and the in-mold pressure limit was increased to 240 bar.

 State aggregation is essential for the algorithm, and the state values were plotted based on this aggregation. The target state range for the filling process, defined as 97–102%, was indicated separately with dotted lines in Fig. 11. This approach guides the algorithm to adjust injection molding parameters to avoid both overfilling and underfilling of the cavity.

Although other state aggregation methods could be applied, it is important that prior knowledge is derived from a comparable aggregation scheme. Fig. 11 illustrates notable differences between the state spaces of various products.

Generally, however, all show a similar trend: at high switch-over volumes, the product is underfilled (well below 100%), while at low switch-over volumes, it is overfilled (well above 100%). Additionally, the acceptable range for filling is wider at higher injection flow rates (>30 cm³/s) compared to lower rates (<30 cm³/s).

> State Space for the Holding Phase

The state value for optimizing the holding phase is more straightforward to define. It relies solely on the measured product weight, which is easy to automate and integrate with the injection molding machine and weighing system.

The state value is calculated as the ratio of the actual part weight produced in each actual cycle (m_actual) to the target weight (m_goal), as described in function (6), where S _hold  represents the holding state. The state aggregation for each product used in the experiments is presented in Fig. 12. Unlike the filling phase, the target aggregated state for holding is symmetric, reflecting the goal of producing a part within a specified tolerance of ±0.5% of the target weight.

> Limitations of the Algorithm

The main strength of the presented algorithm is its ability to learn how to operate within a specific environment and then apply that knowledge to a new, somewhat similar—but still different—environment.

However, this capability also introduces a significant drawback. When the algorithm attempts to transfer knowledge to a substantially different environment, it often makes suboptimal decisions early in the learning process because the new situation requires different actions than those learned previously.

This challenge can arise, for example, when pre-learning is conducted using non-optimized simulation data, while post-learning uses actual injection molding data. Such discrepancies are common if the simulation does not accurately capture factors like material properties, mold geometry, or processing conditions.

Experimental results showed that small variations in product geometry, material, or gate design typically do not cause enough environmental change to prevent the algorithm from functioning effectively.

There are additional constraints as well. The algorithm must maintain the same number of aggregated states and action sets during both pre-learning and post-learning. Moreover, the specific definitions of these aggregated states and actions must remain consistent between the two phases.

For instance, if one aggregated state corresponds to 90–100% filled products during pre-learning but to 10–20% filled products during post-learning, the algorithm will fail to perform well.

Similarly, changes in the range or meaning of actions between pre- and post-learning will require the algorithm to relearn the effects of these new actions from scratch.

When the learning parameter (α) is increased, the weights within each function (𝑤 and θ) can undergo more significant changes. Consequently, using a much larger learning parameter during post-learning compared to pre-learning can cause the algorithm to favor suboptimal choices. This often prevents it from identifying the optimal combination of injection molding settings.

Conversely, if the learning parameter is set too low during post-learning, the algorithm adapts very slowly to changes in the environment. To ensure stable and effective performance, it is recommended to use the same learning parameter for both pre-learning and post-learning phases.

Results

The experiment is divided into two primary sections: filling phase optimization and holding phase optimization. Each section includes subparts based on the learning approach, namely pre-learning and post-learning.

Pre-learning involves initially training the algorithm to acquire prior knowledge, which it later utilizes during post-learning. Post-learning evaluates how well the algorithm applies the learned knowledge when adjusting injection molding settings for a different product.

Filling Phase Optimization

The objective of filling phase optimization is for the algorithm to learn how to fill the mold cavity within a desired time frame without causing overfilling. To achieve this, the algorithm uses three inputs: the product image, the maximum measured pressure, and the filling time.

During this phase, the algorithm can modify two key parameters: the switch-over volume, which determines the screw’s end position and thus the injected melt volume, and the injection flow rate, which controls the screw speed. Changing the injection flow indirectly adjusts both the filling time and the in-mold pressure.

> Pre-Learning for the Filling Phase

For pre-learning, a 1.2 mm thick plate with a fan gate design (Fig. 1e) was used. Initial injection molding settings were set to a switch-over volume of 22 cm³ and an injection flow rate of 15 cm³/s. These parameters resulted in a cavity that was far from fully filled, with a relatively slow melt flow, ensuring no accidental overfilling or mold damage.

The algorithm was allowed to adjust injection flow by increments of 0, ±5, ±10, or ±15 cm³/s and switch-over volume by 0, ±1, ±2, or ±5 cm³. At each learning step, the algorithm selected one combination of these changes (an action) based on its current policy.

Learning was set to conclude after 4,000 steps. Up to 3,000 steps, the algorithm could reset the settings back to the initial values after a specified number of learning steps.

Various learning scenarios were tested with 0, 10, 20, or 30 restarts within the first 3,000 steps (see Fig. 13a–d). Results showed that within a few hundred steps, the algorithm converged to near-optimal settings, as indicated by state values approaching 100%.

However, once near the optimum, the algorithm tended not to return to less effective states such as the initial setting and its vicinity. While this behavior prevents wasting time on poor settings, it also limits exploration, meaning the algorithm might miss discovering optimal choices near the initial state.

This limitation could slow learning when the algorithm is applied to a similar but new product, as returning to the initial state more often would enable faster convergence to the target (see Fig. 13c and d). The knowledge gained during pre-learning was stored in the 𝑤 and 𝑣 weight vectors, which represent the state and state-action values respectively.

These vectors form the main difference between pre-learning and post-learning: during pre-learning, they were initialized with default values (0.5 for each element), while in post-learning, they were initialized with the average values obtained from 100 pre-learning runs.

> Utilizing Prior Knowledge from a Different Gate Design

The gate design is a crucial aspect of injection mold design. During the filling phase, variations in gate geometry lead to different flow paths and shear stresses within the molten material.

Therefore, it is important to verify whether the algorithm can effectively adapt when applied to products that differ only in gate design. This is tested by conducting post-learning on a product whose only difference from the pre-learning product is the gate configuration (see Fig. 14).

During pre-learning, it was evident that the algorithm required several hundred steps to reach the optimal state (see Fig. 13).

This corresponds to the same number of injection molding cycles, which is time-consuming and results in many defective parts. Pre-learning can be likened to a beginner learning to operate the injection molding machine, whereas post-learning resembles an expert fine-tuning the machine settings.

The outcomes of post-learning based on different pre-learning scenarios are presented in Fig. 15.

Post-learning required significantly fewer steps for the algorithm to approach the optimal state, and the initial improvement in state values was noticeably faster. This trend is illustrated in the distribution plots, showing the number of injection molding cycles needed to produce a part meeting quality standards across 100 scenarios.

The green-shaded areas represent the middle 75% of the data, with the upper quartile (Q3) marked for clarity. The distribution became even narrower when pre-learning involved restarts. Since the difference between various restart strategies was minimal, the prior knowledge from pre-learning with 30 restarts was used for further analysis.

These findings demonstrate that if the algorithm learned from a mold with a different gate design, it could reach the optimal state within 22 cycles in 75% of the cases. The cycle count distribution was right-skewed, indicating that most optimal settings were found quickly in most runs.

Performance could be further enhanced by adjusting the range of possible actions or relaxing the quality requirements, potentially approaching the skill level of a professional technician.

Overall, these results confirm that the learning algorithm can successfully apply prior knowledge to a similar product featuring a different gate design.

> Applying Prior Knowledge from a Product Made with a Different Material

In industrial settings, switching to a different material for product manufacturing may be necessary due to sustainability, cost, or engineering considerations.

Because different materials have distinct properties such as specific volume and melt flow index, the optimal injection molding parameters for one material often differ from those of another.

Therefore, it is crucial to examine whether the algorithm can effectively transfer prior knowledge learned from one material to optimize the filling phase for a product made from a different material.

To test this, the same plate design with a 1.2 mm thickness and fan gate used in pre-learning was molded using PLA instead of ABS.

To evaluate the influence of material change on post-learning, two post-learning scenarios were conducted: one using data from the original ABS material (Fig. 16a), and the other using data from the new PLA material (Fig. 16b).

Comparing these two cases shows that the algorithm performed better with ABS, which was expected since both pre-learning and post-learning used the same material and product.

Nevertheless, the performance with PLA was still very promising—the difference in the upper quartile (Q3) of cycles required between the two materials was only five injection molding cycles, despite the material change. This outcome indicates that the algorithm can successfully utilize prior knowledge across different materials.

> Applying Prior Knowledge from a Product with Different Thickness

In real-world manufacturing, companies often produce similar products that vary in features such as thickness or nominal size. Therefore, it is valuable to assess whether the algorithm can effectively utilize prior knowledge when working with products of different thicknesses.

To investigate this, plates with thicknesses of 1 mm, 1.2 mm, and 2.5 mm (Fig. 17) were molded. The 1.2 mm plate was used for pre-learning, while the thicker and thinner plates were used for post-learning.

Product thickness plays a crucial role during the filling phase because thinner parts may freeze prematurely if the injection flow is too low, although they require less melt to fill the cavity. As a result, the optimal injection molding parameters can vary significantly between products of different thicknesses.

Figure 18 presents the results of post-learning applied to plates with thicknesses of 1 mm and 2.5 mm. Between the two, the optimization for the 1 mm plate was more effective using the given prior knowledge.

For the 2.5 mm thick plate, the search space (Fig. 11 d) undergoes greater variation when the part is overfilled, whereas the changes are less pronounced when the part is underfilled.

This can be attributed to the specified pressure limit, which is set at 50 bar for the 2.5 mm plate (compared to 220 bar for others), meaning that a 5–10 bar difference has a relatively larger impact.

The smaller variation in the underfilled region results from the increased cavity thickness—because a larger cavity volume means the same amount of melt causes less change during filling.

These factors complicate the algorithm’s task of finding the optimal state since identical actions have smaller effects when the part is underfilled and larger effects when it is overfilled.

As shown in Figure 18b, the algorithm begins at a significantly better state value than in other cases.

This is because the algorithm adjusts only the injection flow and switch-over volume, not the prepared shot volume, which is pre-set on the injection molding machine. A larger shot volume means that for the same switch-over volume, more melt enters the cavity.

This study treats the prepared shot volume as part of the dosing phase, which the learning algorithm does not modify, nor does it optimize the mold closing and opening phases.

The only conditions are that the shot volume must exceed the possible switch-over volumes and remain constant throughout the learning process.

> Utilizing Prior Knowledge from a Product with Different Geometry for Filling

While changing the material for a specific product is relatively common in practice, it is even more frequent for companies to develop entirely new products.

In these situations, injection molding simulations are often used to help determine the machine settings. However, simulation software tends to be costly and requires significant expertise to operate effectively.

Additionally, simulation models generally need to be fine-tuned and validated through actual injection molding experiments, which involves substantial time, material, and financial resources.

To potentially replace this resource-intensive process, the effectiveness of the learning algorithm was tested by applying prior knowledge gained from molding a product with one geometry to optimize the filling phase of products with different geometries.

For this analysis, the lid product (Fig. 1b) was chosen for post-learning because it features numerous complex elements absent in the plate product, making it considerably different from the original molded part. This distinction is clearly reflected in the search spaces illustrated in Fig. 11a–f.

The lid also has much thinner sections, causing the in-mold pressure to rise significantly higher compared to the plate. This effect is evident in Fig. 19, where state values exceed 150%.

Holding Phase Optimization

The objective of holding phase optimization is to precisely control the product’s weight.

However, the holding phase also serves other important purposes, such as minimizing shrinkage and sink marks.

With an appropriate measurement system and metric, the algorithm can be adapted to optimize these criteria in a manner similar to how it optimizes product weight.

The holding phase follows the filling phase because, as the injected melt begins to cool and shrink, additional melt is injected into the cavity at a specified holding pressure for a certain holding time.

This step adjusts the product’s weight, but the change is relatively minor compared to the weight variation during filling. During this phase, the algorithm can independently adjust both holding pressure and holding time.

There is a limit to how long the holding time can be for conventional cold runner molds, and exceeding this limit is not advisable.

This limit depends on the gate and mold temperature conditions, as the gate solidifies—referred to as gate freeze-off—after a certain period. Once freeze-off occurs, no more melt can enter the cavity.

This behavior is illustrated in Fig. 12, where no significant change is observed in product weight for a given holding pressure beyond a specific holding time. Although gate freeze-off time was not directly considered in the analysis, injection molding scenarios were run to obtain measurement points afterward.

> Pre-Learning for the Holding Phase

For the holding phase optimization, the ABS lid part (Fig. 1b), featuring a complex geometry with a simple point-like gate, was used in pre-learning.

The algorithm could modify holding pressure by increments of 0, ±25, ±50, or ±100 bar, and holding time by 0, ±0.5, ±1, or ±2 seconds independently. As a result, the algorithm selects from 49 possible combinations (7 × 7 actions) of these setting adjustments according to its policy.

Learning was defined to conclude after 6000 steps. More learning steps were required than in filling phase optimization because the algorithm needed additional time to learn an effective policy.

A total of 100 learning scenarios were performed with similar algorithm settings to those used for filling optimization, including different restart scenarios with 0, 10, 20, or 30 restarts within 3000 steps (Fig. 20a–d).

The initial settings for pre-learning were set at 0 bar holding pressure and 0 seconds holding time for safety reasons. Excessive holding pressure can cause the mold to open, potentially leading to flash defects on the product and damage to the mold.

Unlike the filling phase, it is unclear whether increasing the number of restarts significantly reduces the number of initial learning steps. This may be due to the non-linear characteristics of the search space near the starting settings.

Nevertheless, by the end of the learning process, the algorithm successfully approaches the optimization goal. Because 6000 learning steps represent a considerable amount of injection molding cycles, it is recommended to use a design of experiments approach to generate the data required for pre-learning.

> Using Prior Knowledge from a Different Material

Previously, post-learning results for a new material (PLA) in filling optimization were discussed (see 3.1.3). Naturally, changing the material also affects the optimal settings for the holding phase.

Different materials often require varying processing temperatures, which influence the gate freeze-off times and, consequently, the necessary holding time. Additionally, differences in material shrinkage and weight compensation further impact the process.

Figure 21a presents the post-learning results using the new material. It took significantly fewer learning steps to reach the optimal product weight compared to pre-learning.

The figure also shows the distribution of how many learning steps (or injection molding cycles) the algorithm needed to first produce a part meeting the required weight across 100 learning scenarios.

In 75% of the cases, fewer than 20 learning steps were required. However, producing scrap during these 19 initial cycles can still be unacceptable in some production environments.

From the early phase of the reward function, it is evident that the algorithm selects actions that considerably reduce the error.

Thus, the primary challenge is not the algorithm’s decision-making but rather that the initial injection molding settings are far from the ideal ones needed to produce the target part.

Since part of the learning task is to identify these optimal settings, it makes sense to use the optimal settings found during pre-learning as the initial settings for post-learning.

However, pre-learning consisted of 100 different learning scenarios, each with 6000 learning steps. To address this, for each of these 100 scenarios, the injection molding settings from the last 500 steps were collected, and the most frequently occurring settings were chosen as the initial settings for post-learning.

The results of post-learning with these new initial settings show a clear improvement, as seen in Figure 21b compared to Figure 21a.

Here, it is clear that in 75% of the cases, fewer than four injection molding setting adjustments were sufficient for the algorithm to produce a part meeting the desired quality.

A key limitation of this approach is that the optimal injection molding settings from the product used in pre-learning may differ significantly from those needed for the product in post-learning. Therefore, this method is beneficial only if the new initial settings are closer to the target settings than the default ones.

In summary, this study demonstrated that prior knowledge can be effectively leveraged in the algorithm to optimize the holding phase for products with the same nominal geometry but made from different materials.

> Using Prior Knowledge from Different Geometries

While the use of prior knowledge from different products during the filling phase has already been demonstrated (see Fig. 19), it is equally important to explore whether such knowledge can be applied to optimize the holding phase.

The required compensation during holding can vary between products due to differences in product volume and features such as ribs, tubes, or variations in thickness. Moreover, gate size and design may differ across products, which can influence the optimal holding phase settings.

To investigate this, the impact of prior knowledge on holding phase optimization was studied using products with varying shapes, sizes, and gate geometries.

The learning algorithm was initially trained on the lid product during pre-learning. Subsequently, it was applied to the plate product with a thickness of 1 mm and to smaller lid products during post-learning (Fig. 22).

The plate product uses a film gate, unlike the lid and small lid products, which feature a point-like gate. This difference in gate design can influence how holding time affects the process.

Additionally, the small lid is produced using a 16-cavity mold, meaning that holding pressure and holding time impact 16 parts simultaneously. The small lid is also much smaller and thinner, making both these products quite different from the lid product used for pre-learning.

The results of post-learning are presented in Fig. 23. When starting with the initial injection molding settings (Fig. 23a and 23c), the algorithm reached its target within 20 injection molding cycles for the plate and 16 cycles for the small lid in 75% of the cases.

This performance is reasonable considering these are entirely new products and only the holding phase is being optimized.

To improve efficiency, learning scenarios were also conducted starting from the pre-learned injection molding settings (Fig. 23b and 23d).

These results clearly demonstrate that using prior knowledge as the starting point can significantly reduce the number of cycles required to reach the desired outcome.

Algorithm Performance for Optimization

Earlier sections demonstrated how prior knowledge can be leveraged by the learning algorithm in various scenarios.

In analyzing post-learning results, the focus was on determining how many injection molding cycles the algorithm required to produce a quality product for the first time in a new setting. The performance was evaluated across 100 learning scenarios for each product variation.

From these cases, distributions were created showing the number of learning steps (injection molding cycles) needed for the algorithm to achieve the desired product quality.

This chapter presents a brief comparison of the different post-learning outcomes by examining these distributions to aid in interpretation.

> Comparing the Impact of Different Prior Knowledge on Filling Phase Optimization

For filling phase optimization, prior knowledge was obtained through pre-learning on a plate product with a thickness of 1.2 mm and a fan gate design (Fig. 1e).

The learning algorithm’s performance was then tested using this prior knowledge in producing the same product as well as parts with varying geometries and materials. Table 5 summarizes the key metrics derived from the distributions of these learning scenarios.

Table 5. The summary of the post-learning results for filling phase optimization.

The results indicate that the algorithm performed best when the same product was used for both pre-learning and post-learning.

This outcome is expected since the prior knowledge already contained information closely related to the optimal decisions for that specific product.

The second-best performance occurred when the product geometry remained the same between the two learning phases, but the material changed.

In this scenario, differences during the filling phase mainly stem from the materials’ rheological properties. The longest learning time (93 cycles) was observed when the algorithm applied prior knowledge to a completely new geometry (the lid product).

Here, factors such as in-mold pressure, required filling time, and various product features (like ribs and tubes) likely caused significant variation in how individual injection molding parameters affected the process.

However, it’s important to note that the 93-cycle value is an outlier in the distribution (see Fig. 19) and thus does not fully represent the overall learning differences.

When considering the mean or median values, the distributions reveal no major differences in performance across the various uses of prior knowledge.

> Comparison of the Effect of Different Prior Knowledge for Holding Phase Optimization

For holding phase optimization, prior knowledge was obtained from pre-learning on the ABS lid product (Fig. 1b).

Two approaches were examined: post-learning starting from default settings of 0 bar holding pressure and 0 seconds holding time, and post-learning starting from an initial point defined by pre-learning.

In the first approach (Table 6), it is evident that post-learning was fastest for the small lid product. This result is not surprising, as its search space allows the widest range of setting combinations that still produce a quality product.

This is partly because the injection molding machine fills 16 cavities with identical geometry simultaneously, so the melt effect is distributed among those cavities.

Table 6. Summary of post-learning results for holding phase optimization starting from default initial settings.

Similar trends are observed when the starting point for post-learning is based on pre-learning results (Table 7). In this case, the number of learning cycles is noticeably lower because the algorithm begins its exploration closer to the optimal settings.

However, it is important to note that this starting point is not always closer to the optimum in a new environment. Even with this approach, there is at least one learning scenario that requires a high number of cycles (91 cycles).

This could be explained by the fact that the optimal settings range for the plate product (Fig. 12c) is narrower compared to that of the pre-learned product (Fig. 12a), causing the algorithm to overlook this range during exploration potentially.

Nevertheless, other distribution metrics indicate that this high cycle count is not typical and represents an outlier rather than the norm.

Table 7. The summary of the post-learning results for holding phase optimization with a starting point derived from pre-learning.

Conclusion

This study demonstrates that leveraging prior knowledge to optimize both the filling and holding phases in injection molding is effective. However, the proposed approach is specifically designed for an actor-critic algorithm employing state aggregation and discrete actions.

The learning process involves two key stages: a pre-learning phase, where the algorithm learns to configure the injection molding machine, followed by a post-learning phase, during which the algorithm applies prior knowledge gained from pre-learning to set up the machine for a new product.

For filling phase optimization, a method utilizing pressure measurement combined with image processing was introduced. The analysis revealed that the algorithm can successfully learn to configure the machine for products with varying gate designs, materials, part thicknesses, or geometries.

By incorporating prior knowledge, the algorithm can reliably produce parts meeting quality requirements despite such changes.

The study concluded that, for the filling phase, changes in geometry present a greater challenge to the learning algorithm than changes in material (see Table 5). Nevertheless, the method consistently performs well across all scenarios and is recommended for industrial application.

Additionally, the method can finely adjust product weight by optimizing the holding phase. Results confirmed that the algorithm adapts effectively to products with different gate types, geometries, and materials.

Two different initialization strategies were tested for the learning algorithm: one starting from the original default settings (0 bar holding pressure, 0 s holding time), and the other beginning from a starting point informed by pre-learning. The default settings were chosen for safety considerations (results shown in Table 6).

The investigation also demonstrated the benefit of using a pre-learning derived starting point (prior knowledge), which significantly improved the optimization process. These findings (Table 7) indicate that starting closer to the optimum greatly enhances learning efficiency, with the algorithm’s performance comparable to that of an experienced technician.