This forum uses cookies This forum makes use of cookies to store your login information if you are registered, and your last visit if you are not. Cookies are small text documents stored on your computer; the cookies set by this forum can only be used on this website and pose no security risk. Cookies on this forum also track the specific topics you have read and when you last read them. Please confirm whether you accept or reject these cookies being set.A cookie will be stored in your browser regardless of choice to prevent you being asked this question again. You will be able to change your cookie settings at any time using the link in the footer.

 gausssmooth and 2 color graph Sebastian Staacks Administrator Posts: 319 Threads: 6 Joined: Aug 2018 Institution: RWTH Aachen University 11-16-2020, 04:55 PM Interesting one. I have a few ideas: - It could be limit of the numerical stability of our implementation. Even the strongly visible peak is only a deviation of 0.02% and with a sigma of 20 the algorithm is calculating 60 samples to the right and left of the Gauß distribution, summing up all values and normalizing by the raw sum of these samples from the Gauß curve. Especially the small values at the edge will be close to the resolution of the floating point numbers when added to the sum, so something might be happening there... - If could be that the peak occurs exactly 60 samples before the end of the measurement. As mentioned above, we calculate a width of 3*sigma=60 samples to get numerically close to a true Gauss, which otherwise needs to be calculated across all samples. When we do not have enough samples to the right (or left) we need to change the strategy and instead of using a pre-calculated normalization value, we need to normalize with the values that match the available data. Again, I do not see it right away in the code, but this will also have a numerical limit. Could you check if it is 60 samples to the end? By the way: Have you tried the LOESS smoothing? Not sure if it is more stable though. « Next Oldest | Next Newest »

 Messages In This Thread gausssmooth and 2 color graph - by solid - 11-14-2020, 09:35 PM RE: gausssmooth and 2 color graph - by solid - 11-15-2020, 12:29 AM RE: gausssmooth and 2 color graph - by solid - 11-15-2020, 10:37 AM RE: gausssmooth and 2 color graph - by Sebastian Staacks - 11-16-2020, 04:55 PM RE: gausssmooth and 2 color graph - by solid - 11-16-2020, 08:04 PM RE: gausssmooth and 2 color graph - by Jens Noriʇzsɔɥ - 11-16-2020, 10:32 PM RE: gausssmooth and 2 color graph - by Sebastian Staacks - 11-16-2020, 10:51 PM RE: gausssmooth and 2 color graph - by Jens Noriʇzsɔɥ - 11-17-2020, 12:04 AM RE: gausssmooth and 2 color graph - by Sebastian Staacks - 11-16-2020, 10:50 PM RE: gausssmooth and 2 color graph - by solid - 11-17-2020, 12:24 AM RE: gausssmooth and 2 color graph - by Sebastian Staacks - 11-25-2020, 06:40 PM RE: gausssmooth and 2 color graph - by solid - 11-27-2020, 12:41 PM RE: gausssmooth and 2 color graph - by Sebastian Staacks - 11-27-2020, 01:14 PM RE: gausssmooth and 2 color graph - by solid - 11-27-2020, 03:13 PM RE: gausssmooth and 2 color graph - by solid - 04-10-2021, 01:30 PM RE: gausssmooth and 2 color graph - by solid - 04-10-2021, 08:19 PM RE: gausssmooth and 2 color graph - by solid - 04-15-2021, 08:44 PM RE: gausssmooth and 2 color graph - by Sebastian Staacks - 04-15-2021, 08:49 PM RE: gausssmooth and 2 color graph - by solid - 04-15-2021, 09:11 PM RE: gausssmooth and 2 color graph - by Jens Noriʇzsɔɥ - 04-16-2021, 10:04 AM RE: gausssmooth and 2 color graph - by Jens Noriʇzsɔɥ - 04-16-2021, 09:34 PM

Forum Jump: