I'm having a real hard time with a formatted search and I figured out that it has to do with the decimal precision. Two values that are multiplied together, if one is a decimal and the other is a number >= 1.0, it works. If both values are < 1.0, it has problems. If I add the two numbers together that are < 1.0, the result is fine. If I multiple two numbers together that are < 1.0 it doesn't. The query below demonstrates this.
declare @OrderTotal float
declare @OrderExp float
declare @DiscPct float
set @OrderTotal = .75
set @DiscPct = 2
set @DiscPct = @DiscPct / 100
set @OrderTotal = @OrderTotal * @DiscPct
select @OrderTotal
Result is 1.5 instead of .15 (.02 x .75)
If one changes the .75 to 1.0, the result is correct a .02
I played around with Float and Real, but I don't know enough about either. Any help would be greatly appreciated.
Request clarification before answering.
.02 x .75 is 0.015 and not 0.15 as you wrote. I run this query and the result is correct, so I think that it belongs to your sql settings.
Try to change it as
declare @OrderTotal numeric(9,6)
declare @OrderExp numeric(9,6)
declare @DiscPct numeric(9,6)
set @OrderTotal = .75
set @DiscPct = 2
set @DiscPct = @DiscPct / 100
set @OrderTotal = convert(numeric(9,6), @OrderTotal * @DiscPct)
select @OrderTotal
and let me know, if it works
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
| User | Count |
|---|---|
| 14 | |
| 13 | |
| 7 | |
| 6 | |
| 5 | |
| 4 | |
| 3 | |
| 2 | |
| 2 | |
| 1 |
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.