Quantcast
Channel: MATLAB Central Newsreader Recent Posts
Viewing all articles
Browse latest Browse all 19628

Precision limit of sqrt(x)

$
0
0
Hi,

I want to compute the sqrt of a fixed point data and below is the sample code.

x = ufi(4092/4096,12,12); % 0.9990 in 0Q12 format
y = ufi(0,12,12);
y(:) = sqrt(x);

The result is y = 0.9844 in 0Q12 format. I just had a floating point implementation which yields a result of y = 0.9995. As we can see, the error is quite. Is there a way to increase the accuracy of sqrt ?



 

Viewing all articles
Browse latest Browse all 19628

Trending Articles