top of page

Rounding or truncating to a specific number of decimal places

Suggested by Keith Recker of GMT Corporation

There are times when a custom macro programmer needs to round a calculated value to a given number of decimal places. Say for example, a given variable is going to represent something that only requires (or allows) a two decimal place format. Having more decimal places will interfere with the functionality or use-ability of the variable.

This was Keith Recker’s problem. He required a two place decimal format: “We bore a hole and then probe it’s depth. The depth dimension of the bore depth is 42.30 +/-.15mm. All I want to engrave is 42.XX (note again the two place format). Since the probe checks the depth, there’s normally a reading of more than two places, like 42.3053 or 42.3073, The bore depth is not that critical so we do not need to go out 3 or 4 decimal places and engrave an extra number/s. Once this part gets assembled, the line operator will look at the engraved dimension (say 42.36), then they’ll grab the desired shim pack needed, lay it at the bottom of the bore, then install a bearing.”

With the DPRNT command (used to send data through the communications port), it is possible to specify the number of decimal places before and after the decimal point. However, the actual value of the variable being transmitted will remain the same. So the DPRNT command will not help with this problem.

Some computer programming languages have functions that allow the easy manipulation of decimal places for variables, but custom macro B has no such “easy” function. With custom macro B, much more ingenuity is required.

There are, of course, two custom macro B functions that provide truncation (FIX) and rounding (ROUND). But these functions force the result to be a whole (integer) number. While these functions will be used in our algorithm for truncating and rounding decimal places, by themselves they do not achieve the intended goal.

Here is an example calling command that we’ll use to get the results we want. It is pretty flexible, allowing you to specify the value to be manipulated, whether you want the value truncated or rounded, the number of decimal places (after the decimal point), and the common variable number in which the calculated value will be stored.

  • N050 G65 P1000 S#121 M1.0 D2.0 V100.0

  • S: The value to be manipulated. It is a variable that has been attained by a prior calculation – and could have any number of decimal places after the decimal point.

  • M: This specifies how the value will be manipulated. If set to 0 (M0.0), the value will be truncated. If set to 1.0 (M1.0), the value will be rounded.

  • D: This is the number of decimal places we desire in the resulting value.

  • V: The variable number in which the result will be stored. In our example, the result will be stored in common variable #100.

The calling command in line N050 is our “wouldn’t it be nice if…” command. It lets us set some goals for the custom macro. With the current settings, we’re saying “Wouldn’t it be nice if our custom macro would take the value of #121 (we’ll say it is currently 44.1783) and round it (not truncate it) to two decimal places after the decimal point and store result in variable #100?” With #121 set to 44.1783, the value being stored in #100 will be 44.18. Note that if M is set to zero in the calling command, the value will be truncated to two decimal places, meaning the resulting value in #100 will be 44.17.

A note about returning a value from the custom macro - Even if you don’t have a need to manipulate decimal places, the technique we use to return a calculated value will be helpful whenever you need your custom macros to calculate a value needed back in the main program. It lets the user specify which common variable will be used. And since common variables will be retained from custom macro to custom macro, the calculated value will be available for use in the calling program (after line N050 in our example).

Now comes the hard part. We have to come up with a way to round or truncate to the specified number of decimal places.

While you may be able to come up with a better/shorter method, here is one method that works. Note that we begin by obtaining the whole number value for the value. We then start working on the decimal portion, multiplying it time 1, or 10, or 100, and so on to attain the appropriate number of decimal places as a whole number value. We then round or truncate again to get just the whole number value for this amount. We divide this result by the same value as we multiplied by before to get back to the decimal portion. And finally, we add the decimal portion to the whole number portion. Again, you may be able to come up with a better algorithm.

Here is the custom macro. The notes in the macro follow along with what happens form our example calling command, with 44.1783 as the value to round to two decimal places. Local variable representations for letter address arguments are S: #19, M: #13, D: #7, V: #22.

  • O1000 (Custom macro to manipulate decimal places)

  • #1 = FIX[#19] (Attain whole number portion: #1 is 44)

  • #2 = #19 - #1 (Get decimal portion: #2 is 0.1783)

  • #11 = 1 (Begin loop to determine number to multiply: it will be #10)

  • #10 = 1

  • N1 IF [#11 GT #7] GOTO 5

  • #10 = #10 * 10

  • #11 = #11+1

  • GOTO 1

  • N5 #3 = #2 * #10 (Get decimal places value as whole number portion: #3 is 17.83)

  • IF [#13 EQ 0] GOTO 6

  • GOTO 10

  • N6 #4 = FIX[#3] (Truncate to decimal places: #4 is 17)

  • GOTO 30

  • N10 IF [#13 EQ 1.0] GOTO 20

  • GOTO 25

  • N20 #4 = ROUND [#3] (Round to decimal places: #4 is 18)

  • GOTO 30

  • N25 #3000 = 100 (M MUST BE 0 or 1)

  • N30 #5 = #4 /#10 (Get back to decimal format: #5 is 0.17 or 0.18)

  • #[#22] = #1 + #5 (This is the format you want: #100 is 44.17 or 44.18)

  • M99 (End of custom macro)

1,120 views0 comments

Recent Posts

See All


bottom of page