[ About | Services | Products | Search | Feedback | Support ]

PowerBuilder Tips

IsNull function loses precision

Back to the Tips

---

Today we found out that the IsNull function in SQL Anywhere can cause loss of precision.  Below is the code which caused the problem and an example of the changed output.

The following is the line which caused the problem:

set @expected_average_inflow=isnull(@expected_average_inflow,0);

And we changed to:

if @actual_average_inflow is null then
  set @actual_average_inflow=0.0
end if

The first line produced the following result:

149.592593

The solution produced the following

149.59259259999996

Which, for our situation, is quite important.  If I recall correctly, you need to cast this to a double first to get it to work.  I'm tracking down the actual solution, which Sybase are fixing, and will post it here.

Back to the Tips

---

[ About | Services | Products | Search | Feedback | Support ]