EPICS Base

Bug #1743795
Comment #6

Comment 6 for bug 1743795

Revision history for this message

Heinz Junkes (junkes) wrote on 2018-07-11: Re: [Bug 1743795] Re: Compress record FIFO/LIFO not working

Hi Andrew,

Ok, this happens with nsam < 16 :-(

In the code for lifo-mode there is line

offset = (offset-1) % nsam;

offset is epicsUInt32.

that does not give comprehensible results.

e.g.
dumm.c

#include <stdio.h>
#include <stdint.h>

int main(int argc, char *argv[])
{

uint32_t offset;

int nsam1 = 10;
int nsam2 = 16;

offset = 0;
offset = (offset-1) % nsam1;
printf(" offset 1 = %d\n", offset);

offset = 0;
offset = (offset-1) % nsam2;
printf(" offset 2 = %d\n", offset);
}

h1:~ heinz$ ./dumm
offset 1 = 5
offset 2 = 15

I could write
if (offset == 0) offset = nsam-1

I found no explanation why from 16 it is right.
Have you ever seen this elsewhere?

Heinz
------------------------------------------------------------------------------
Fritz-Haber-Institut | Phone: (+49 30) 8413-4270
Heinz Junkes | Fax (G3+G4): (+49 30) 8413-5900
Faradayweg 4-6 |
D - 14195 Berlin | E-Mail: <email address hidden>
------------------------------------------------------------------------------

> On 27. Jun 2018, at 00:04, Andrew Johnson <email address hidden> wrote:
>
> I'm not convinced this patch is quite right yet. After applying it I'm
> seeing zeros instead of the values fetched from the input. I used a
> slightly simpler database and set ring.SCAN to "1 second" by hand after
> connecting the camonitor:
>
> record(calc,"counter") {
> field(CALC, "VAL+1")
> }
> record(compress,"ring") {
> # field(SCAN, "1 second")
> field(NSAM, "10")
> field(ALG, "Circular Buffer")
> field(INP, "counter PP")
> # field(BALG, "FIFO Buffer")
> field(BALG, "LIFO Buffer")
> field(PREC, "1")
> }
>
> This is the camonitor output when ring.BALG is "LIFO Buffer":
>
> ring 2018-06-26 16:37:09.663332 1
> ring 2018-06-26 16:37:10.663222 2 2 1
> ring 2018-06-26 16:37:11.663231 3 3 2 1
> ring 2018-06-26 16:37:12.663253 4 4 3 2 1
> ring 2018-06-26 16:37:13.663229 5 5 4 3 2 1
> ring 2018-06-26 16:37:14.663222 6 6 5 4 3 2 1
> ring 2018-06-26 16:37:15.663239 7 7 0 0 0 0 6 5
> ring 2018-06-26 16:37:16.663247 8 8 7 0 0 0 0 6 5
> ring 2018-06-26 16:37:17.663212 9 9 8 7 0 0 0 0 6 5
> ring 2018-06-26 16:37:18.663218 10 10 9 8 7 0 0 0 0 6 5
> ring 2018-06-26 16:37:19.663245 10 11 10 9 8 7 0 0 0 0 6
> ring 2018-06-26 16:37:20.663221 10 12 11 10 9 8 7 0 0 0 0
> ring 2018-06-26 16:37:21.663227 10 13 0 0 0 0 12 11 10 9 8
> ring 2018-06-26 16:37:22.663223 10 14 13 0 0 0 0 12 11 10 9
> ring 2018-06-26 16:37:23.663220 10 15 14 13 0 0 0 0 12 11 10
> ring 2018-06-26 16:37:24.663224 10 16 15 14 13 0 0 0 0 12 11
> ring 2018-06-26 16:37:25.663230 10 17 16 15 14 13 0 0 0 0 12
> ring 2018-06-26 16:37:26.663220 10 18 17 16 15 14 13 0 0 0 0
> ring 2018-06-26 16:37:27.663257 10 19 0 0 0 0 18 17 16 15 14
> ring 2018-06-26 16:37:28.663227 10 20 19 0 0 0 0 18 17 16 15
>
> This is the camonitor output when ring.BALG is "FIFO Buffer":
>
> ring 2018-06-26 16:33:59.111461 0
> ring 2018-06-26 16:34:00.111415 2 0 0
> ring 2018-06-26 16:34:01.111395 3 0 0 0
> ring 2018-06-26 16:34:02.111400 4 0 0 0 0
> ring 2018-06-26 16:34:03.111396 5 0 0 0 0 0
> ring 2018-06-26 16:34:04.111428 6 0 0 0 0 1 2
> ring 2018-06-26 16:34:05.111413 7 0 0 0 1 2 3 4
> ring 2018-06-26 16:34:06.111410 8 0 0 1 2 3 4 5 6
> ring 2018-06-26 16:34:07.111398 9 0 1 2 3 4 5 6 7 8
> ring 2018-06-26 16:34:08.111426 10 1 2 3 4 5 6 7 8 9 10
>
> For the FIFO case it works Okay beyond the rows shown, whereas for the
> LIFO case I always get 4 adjacent zero values in the buffer, e.g:
>
> ring 2018-06-26 16:45:31.145797 10 199 0 0 0 0 198 197 196 195 194
> ring 2018-06-26 16:45:32.145824 10 200 199 0 0 0 0 198 197 196 195
> ring 2018-06-26 16:45:33.145803 10 201 200 199 0 0 0 0 198 197 196
> ring 2018-06-26 16:45:34.145786 10 202 201 200 199 0 0 0 0 198 197
> ring 2018-06-26 16:45:35.145771 10 203 202 201 200 199 0 0 0 0 198
> ring 2018-06-26 16:45:36.145814 10 204 203 202 201 200 199 0 0 0 0
> ring 2018-06-26 16:45:37.145809 10 205 0 0 0 0 204 203 202 201 200
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1743795
>
> Title:
> Compress record FIFO/LIFO not working
>
> Status in EPICS Base:
> In Progress
> Status in EPICS Base 3.16 series:
> In Progress
> Status in EPICS Base 7.0 series:
> In Progress
>
> Bug description:
> Reported by Daniel Schick <daniel.schick_AT_mbi-berlin.de>:
>
> I am currently using epics base 3.16.1 and I have a problem with the compress record used as ring buffer.
> In both, Epics 3.16.1 and 7.0 the update of the ring buffer is not FIFO or LIFO but happens somewhere in the middle of the array.
> When I use the latest 3.15 branch everything works fine.
>
> Here is an example of my test db-file and the output from camonitor:
>
> record(calc,"counter") {
> field(CALC,"VAL+1")
> field(SCAN,"1 second")
> }
>
> record(compress,"ring") {
> field(DESC, "test")
> field(NSAM,"10")
> field(ALG,"Circular Buffer")
> field(INP,"counter")
> field(SCAN,"1 second")
> field(BALG,"LIFO")
> field(PREC,"1")
> }
>
> with epics 3.15 it is fine:
>
> ~ $ camonitor ring
> ring *** Not connected (PV not found)
> ring 2018-01-08 12:55:16.628324 8 0 1 2 3 4 5 6 7
> ring 2018-01-08 12:55:17.628331 9 0 1 2 3 4 5 6 7 8
> ring 2018-01-08 12:55:18.628323 10 0 1 2 3 4 5 6 7 8 9
> ring 2018-01-08 12:55:19.628330 10 1 2 3 4 5 6 7 8 9 10
> ring 2018-01-08 12:55:20.628326 10 2 3 4 5 6 7 8 9 10 11
> ring 2018-01-08 12:55:21.628331 10 3 4 5 6 7 8 9 10 11 12
> ring 2018-01-08 12:55:22.628327 10 4 5 6 7 8 9 10 11 12 13
> ring 2018-01-08 12:55:23.628324 10 5 6 7 8 9 10 11 12 13 14
> ring 2018-01-08 12:55:24.628325 10 6 7 8 9 10 11 12 13 14 15
> ring 2018-01-08 12:55:25.628332 10 7 8 9 10 11 12 13 14 15 16
> ring 2018-01-08 12:55:26.628326 10 8 9 10 11 12 13 14 15 16 17
> ring 2018-01-08 12:55:27.628330 10 9 10 11 12 13 14 15 16 17 18
> ring 2018-01-08 12:55:28.628323 10 10 11 12 13 14 15 16 17 18 19
> ring 2018-01-08 12:55:29.628325 10 11 12 13 14 15 16 17 18 19 20
> ring 2018-01-08 12:55:30.628327 10 12 13 14 15 16 17 18 19 20 21
> ring 2018-01-08 12:55:31.628331 10 13 14 15 16 17 18 19 20 21 22
> ring 2018-01-08 12:55:32.628327 10 14 15 16 17 18 19 20 21 22 23
>
>
> with epics 3.16 and 7.0 I get the weird update behaviour:
>
> ~ $ camonitor ring
> ring *** Not connected (PV not found)
> ring 2018-01-08 14:23:50.889595 2 0 1
> ring 2018-01-08 14:23:51.889598 3 0 1 2
> ring 2018-01-08 14:23:52.889594 4 0 1 2 3
> ring 2018-01-08 14:23:53.889596 5 0 1 2 3 4
> ring 2018-01-08 14:23:54.889603 6 0 1 2 3 4 5
> ring 2018-01-08 14:23:55.889601 7 0 1 2 3 4 5 6
> ring 2018-01-08 14:23:56.889594 8 0 1 2 3 4 5 6 7
> ring 2018-01-08 14:23:57.889598 9 0 1 2 3 4 5 6 7 8
> ring 2018-01-08 14:23:58.889592 10 6 7 8 9 0 1 2 3 4 5
> ring 2018-01-08 14:23:59.889607 10 7 8 9 10 1 2 3 4 5 6
> ring 2018-01-08 14:24:00.889598 10 8 9 10 11 2 3 4 5 6 7
> ring 2018-01-08 14:24:01.889599 10 9 10 11 12 3 4 5 6 7 8
> ring 2018-01-08 14:24:02.889596 10 10 11 12 13 4 5 6 7 8 9
> ring 2018-01-08 14:24:03.889594 10 11 12 13 14 5 6 7 8 9 10
> ring 2018-01-08 14:24:04.889598 10 12 13 14 15 6 7 8 9 10 11
> ring 2018-01-08 14:24:05.889596 10 13 14 15 16 7 8 9 10 11 12
> ring 2018-01-08 14:24:06.889597 10 14 15 16 17 8 9 10 11 12 13
> ring 2018-01-08 14:24:07.889598 10 15 16 17 18 9 10 11 12 13 14
> ring 2018-01-08 14:24:08.889595 10 16 17 18 19 10 11 12 13 14 15
> ring 2018-01-08 14:24:09.889616 10 17 18 19 20 11 12 13 14 15 16
> ring 2018-01-08 14:24:10.889598 10 18 19 20 21 12 13 14 15 16 17
> ring 2018-01-08 14:24:11.889599 10 19 20 21 22 13 14 15 16 17 18
> ring 2018-01-08 14:24:12.889595 10 20 21 22 23 14 15 16 17 18 19
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/epics-base/+bug/1743795/+subscriptions

Hi Andrew,

Ok, this happens with nsam < 16 :-(

In the code for lifo-mode there is line

offset = (offset-1) % nsam;

offset is epicsUInt32.

that does not give comprehensible results.

e.g.
dumm.c

#include <stdio.h>
#include <stdint.h>

int main(int argc, char *argv[])
{

uint32_t offset;

int nsam1 = 10;
int nsam2 = 16;

offset = 0;
offset = (offset-1) % nsam1;
printf(" offset 1 = %d\n", offset);

offset = 0;
offset = (offset-1) % nsam2;
printf(" offset 2 = %d\n", offset);
}

h1:~ heinz$ ./dumm
 offset 1 = 5
 offset 2 = 15

I could write
if (offset == 0) offset =  nsam-1

I found no explanation why from 16 it is right.
Have you ever seen this elsewhere?

Heinz
------------------------------------------------------------------------------
Fritz-Haber-Institut    | Phone:         (+49 30) 8413-4270
Heinz Junkes             | Fax (G3+G4):   (+49 30) 8413-5900
Faradayweg 4-6        | 
D - 14195 Berlin        | E-Mail:        junkes@fhi-berlin.mpg.de
------------------------------------------------------------------------------

> On 27. Jun 2018, at 00:04, Andrew Johnson <anj@aps.anl.gov> wrote:
> 
> I'm not convinced this patch is quite right yet. After applying it I'm
> seeing zeros instead of the values fetched from the input. I used a
> slightly simpler database and set ring.SCAN to "1 second" by hand after
> connecting the camonitor:
> 
> record(calc,"counter") {
> field(CALC, "VAL+1")
> }
> record(compress,"ring") {
> # field(SCAN, "1 second")
> field(NSAM, "10")
> field(ALG, "Circular Buffer")
> field(INP, "counter PP")
> # field(BALG, "FIFO Buffer")
> field(BALG, "LIFO Buffer")
> field(PREC, "1")
> }
> 
> This is the camonitor output when ring.BALG is "LIFO Buffer":
> 
> ring                           2018-06-26 16:37:09.663332 1  
> ring 2018-06-26 16:37:10.663222 2 2 1  
> ring 2018-06-26 16:37:11.663231 3 3 2 1  
> ring 2018-06-26 16:37:12.663253 4 4 3 2 1  
> ring 2018-06-26 16:37:13.663229 5 5 4 3 2 1  
> ring 2018-06-26 16:37:14.663222 6 6 5 4 3 2 1  
> ring 2018-06-26 16:37:15.663239 7 7 0 0 0 0 6 5  
> ring 2018-06-26 16:37:16.663247 8 8 7 0 0 0 0 6 5  
> ring 2018-06-26 16:37:17.663212 9 9 8 7 0 0 0 0 6 5  
> ring 2018-06-26 16:37:18.663218 10 10 9 8 7 0 0 0 0 6 5  
> ring 2018-06-26 16:37:19.663245 10 11 10 9 8 7 0 0 0 0 6  
> ring 2018-06-26 16:37:20.663221 10 12 11 10 9 8 7 0 0 0 0  
> ring 2018-06-26 16:37:21.663227 10 13 0 0 0 0 12 11 10 9 8  
> ring 2018-06-26 16:37:22.663223 10 14 13 0 0 0 0 12 11 10 9  
> ring 2018-06-26 16:37:23.663220 10 15 14 13 0 0 0 0 12 11 10  
> ring 2018-06-26 16:37:24.663224 10 16 15 14 13 0 0 0 0 12 11  
> ring 2018-06-26 16:37:25.663230 10 17 16 15 14 13 0 0 0 0 12  
> ring 2018-06-26 16:37:26.663220 10 18 17 16 15 14 13 0 0 0 0  
> ring 2018-06-26 16:37:27.663257 10 19 0 0 0 0 18 17 16 15 14  
> ring 2018-06-26 16:37:28.663227 10 20 19 0 0 0 0 18 17 16 15  
> 
> This is the camonitor output when ring.BALG is "FIFO Buffer":
> 
> ring                           2018-06-26 16:33:59.111461 0  
> ring 2018-06-26 16:34:00.111415 2 0 0  
> ring 2018-06-26 16:34:01.111395 3 0 0 0  
> ring 2018-06-26 16:34:02.111400 4 0 0 0 0  
> ring 2018-06-26 16:34:03.111396 5 0 0 0 0 0  
> ring 2018-06-26 16:34:04.111428 6 0 0 0 0 1 2  
> ring 2018-06-26 16:34:05.111413 7 0 0 0 1 2 3 4  
> ring 2018-06-26 16:34:06.111410 8 0 0 1 2 3 4 5 6  
> ring 2018-06-26 16:34:07.111398 9 0 1 2 3 4 5 6 7 8  
> ring 2018-06-26 16:34:08.111426 10 1 2 3 4 5 6 7 8 9 10  
> 
> For the FIFO case it works Okay beyond the rows shown, whereas for the
> LIFO case I always get 4 adjacent zero values in the buffer, e.g:
> 
> ring 2018-06-26 16:45:31.145797 10 199 0 0 0 0 198 197 196 195 194  
> ring 2018-06-26 16:45:32.145824 10 200 199 0 0 0 0 198 197 196 195  
> ring 2018-06-26 16:45:33.145803 10 201 200 199 0 0 0 0 198 197 196  
> ring 2018-06-26 16:45:34.145786 10 202 201 200 199 0 0 0 0 198 197  
> ring 2018-06-26 16:45:35.145771 10 203 202 201 200 199 0 0 0 0 198  
> ring 2018-06-26 16:45:36.145814 10 204 203 202 201 200 199 0 0 0 0  
> ring 2018-06-26 16:45:37.145809 10 205 0 0 0 0 204 203 202 201 200
> 
> -- 
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1743795
> 
> Title:
>  Compress record FIFO/LIFO not working
> 
> Status in EPICS Base:
>  In Progress
> Status in EPICS Base 3.16 series:
>  In Progress
> Status in EPICS Base 7.0 series:
>  In Progress
> 
> Bug description:
>  Reported by Daniel Schick <daniel.schick_AT_mbi-berlin.de>:
> 
>  I am currently using epics base 3.16.1 and I have a problem with the compress record used as ring buffer.
>  In both, Epics 3.16.1 and 7.0 the update of the ring buffer is not FIFO or LIFO but happens somewhere in the middle of the array.
>  When I use the latest 3.15 branch everything works fine.
> 
>  Here is an example of my test db-file and the output from camonitor:
> 
>  record(calc,"counter") {
>   field(CALC,"VAL+1")
>   field(SCAN,"1 second")
>  }
> 
>  record(compress,"ring") {
>   field(DESC, "test")
>   field(NSAM,"10")
>   field(ALG,"Circular Buffer")
>   field(INP,"counter")
>   field(SCAN,"1 second")
>   field(BALG,"LIFO")
>   field(PREC,"1")
>  }
> 
>  with epics 3.15 it is fine:
> 
>   ~ $ camonitor ring
>  ring *** Not connected (PV not found)
>  ring 2018-01-08 12:55:16.628324 8 0 1 2 3 4 5 6 7 
>  ring 2018-01-08 12:55:17.628331 9 0 1 2 3 4 5 6 7 8 
>  ring 2018-01-08 12:55:18.628323 10 0 1 2 3 4 5 6 7 8 9 
>  ring 2018-01-08 12:55:19.628330 10 1 2 3 4 5 6 7 8 9 10 
>  ring 2018-01-08 12:55:20.628326 10 2 3 4 5 6 7 8 9 10 11 
>  ring 2018-01-08 12:55:21.628331 10 3 4 5 6 7 8 9 10 11 12 
>  ring 2018-01-08 12:55:22.628327 10 4 5 6 7 8 9 10 11 12 13 
>  ring 2018-01-08 12:55:23.628324 10 5 6 7 8 9 10 11 12 13 14 
>  ring 2018-01-08 12:55:24.628325 10 6 7 8 9 10 11 12 13 14 15 
>  ring 2018-01-08 12:55:25.628332 10 7 8 9 10 11 12 13 14 15 16 
>  ring 2018-01-08 12:55:26.628326 10 8 9 10 11 12 13 14 15 16 17 
>  ring 2018-01-08 12:55:27.628330 10 9 10 11 12 13 14 15 16 17 18 
>  ring 2018-01-08 12:55:28.628323 10 10 11 12 13 14 15 16 17 18 19 
>  ring 2018-01-08 12:55:29.628325 10 11 12 13 14 15 16 17 18 19 20 
>  ring 2018-01-08 12:55:30.628327 10 12 13 14 15 16 17 18 19 20 21 
>  ring 2018-01-08 12:55:31.628331 10 13 14 15 16 17 18 19 20 21 22 
>  ring 2018-01-08 12:55:32.628327 10 14 15 16 17 18 19 20 21 22 23
> 
> 
>  with epics 3.16 and 7.0 I get the weird update behaviour:
> 
>  ~ $ camonitor ring
>  ring *** Not connected (PV not found)
>  ring 2018-01-08 14:23:50.889595 2 0 1 
>  ring 2018-01-08 14:23:51.889598 3 0 1 2 
>  ring 2018-01-08 14:23:52.889594 4 0 1 2 3 
>  ring 2018-01-08 14:23:53.889596 5 0 1 2 3 4 
>  ring 2018-01-08 14:23:54.889603 6 0 1 2 3 4 5 
>  ring 2018-01-08 14:23:55.889601 7 0 1 2 3 4 5 6 
>  ring 2018-01-08 14:23:56.889594 8 0 1 2 3 4 5 6 7 
>  ring 2018-01-08 14:23:57.889598 9 0 1 2 3 4 5 6 7 8 
>  ring 2018-01-08 14:23:58.889592 10 6 7 8 9 0 1 2 3 4 5 
>  ring 2018-01-08 14:23:59.889607 10 7 8 9 10 1 2 3 4 5 6 
>  ring 2018-01-08 14:24:00.889598 10 8 9 10 11 2 3 4 5 6 7 
>  ring 2018-01-08 14:24:01.889599 10 9 10 11 12 3 4 5 6 7 8 
>  ring 2018-01-08 14:24:02.889596 10 10 11 12 13 4 5 6 7 8 9 
>  ring 2018-01-08 14:24:03.889594 10 11 12 13 14 5 6 7 8 9 10 
>  ring 2018-01-08 14:24:04.889598 10 12 13 14 15 6 7 8 9 10 11 
>  ring 2018-01-08 14:24:05.889596 10 13 14 15 16 7 8 9 10 11 12 
>  ring 2018-01-08 14:24:06.889597 10 14 15 16 17 8 9 10 11 12 13 
>  ring 2018-01-08 14:24:07.889598 10 15 16 17 18 9 10 11 12 13 14 
>  ring 2018-01-08 14:24:08.889595 10 16 17 18 19 10 11 12 13 14 15 
>  ring 2018-01-08 14:24:09.889616 10 17 18 19 20 11 12 13 14 15 16 
>  ring 2018-01-08 14:24:10.889598 10 18 19 20 21 12 13 14 15 16 17 
>  ring 2018-01-08 14:24:11.889599 10 19 20 21 22 13 14 15 16 17 18 
>  ring 2018-01-08 14:24:12.889595 10 20 21 22 23 14 15 16 17 18 19
> 
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/epics-base/+bug/1743795/+subscriptions