The SIO2 project
  1. The SIO2 project
  2. SIO-1284

filetracker-server support for logging to stderr

        Details

        • Type: Improvement Improvement
        • Status: Closed Closed
        • Priority: Major Major
        • Resolution: Fixed
        • Affects Version/s: Current Version
        • Fix Version/s: None
        • Component/s: Evaluation Engine / Workers
        • Labels:
          None
        • Sprint:
          The First Sprint, The Second Sprint

          Description

          This is needed to use the supervisord's log rotation facility out of the box. For now filetracker logs are not rotated and it's not good...

            Activity

            Ascending order - Click to sort in descending order
            Szymon Acedański made changes -
            Field Original Value New Value
            Status New [ 10000 ] Open [ 1 ]
            Szymon Acedański made changes -
            Fix Version/s The First Sprint [ 11500 ]
            Szymon Acedański made changes -
            Assignee Marek Bardoński [ bdfhjk ]
            Szymon Acedański made changes -
            Sprint The First Sprint [ 2 ]
            Marek Bardoński made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            Hide
            Permalink
            Marek Bardoński added a comment -
            filetracker-server is emmiting two kinds of logs, one from standard logger which is going to stdout, another from lighthttpd (using mod_accesslog) is going where the log file is specified, now to the log file

            According to documentation, we can choose the stderr [1] instead of our custom log file by passing the argument

            accesslog <log:string> Defines the log target ("stderr", filename, etc)

            Then the stream will be redirected to stderr, catched by supervisord to rotated log.

            [1] http://redmine.lighttpd.net/projects/4/wiki/Mod_accesslog

            What was done in my change
            Show
            Marek Bardoński added a comment - filetracker-server is emmiting two kinds of logs, one from standard logger which is going to stdout, another from lighthttpd (using mod_accesslog) is going where the log file is specified, now to the log file According to documentation, we can choose the stderr [1] instead of our custom log file by passing the argument accesslog <log:string> Defines the log target ("stderr", filename, etc) Then the stream will be redirected to stderr, catched by supervisord to rotated log. [1] http://redmine.lighttpd.net/projects/4/wiki/Mod_accesslog What was done in my change
            Hide
            Permalink
            Gerrit Gerrit added a comment -
            Change Ic9f378cac19b68ffd47d332d547c60de81b76454, patchset 1
            https://gerrit.sio2project.mimuw.edu.pl/1806

            Filetracker logs redirected to rotated supervisord log. Fix SIO-1284

            Change-Id: Ic9f378cac19b68ffd47d332d547c60de81b76454
            Show
            Gerrit Gerrit added a comment - Change Ic9f378cac19b68ffd47d332d547c60de81b76454, patchset 1 https://gerrit.sio2project.mimuw.edu.pl/1806 Filetracker logs redirected to rotated supervisord log. Fix SIO-1284 Change-Id: Ic9f378cac19b68ffd47d332d547c60de81b76454
            Hide
            Permalink
            Gerrit Gerrit added a comment -
            Change Ic9f378cac19b68ffd47d332d547c60de81b76454, patchset 2
            https://gerrit.sio2project.mimuw.edu.pl/1806

            SIO-1284 filetracker-server support for logging to stderr

            Change-Id: Ic9f378cac19b68ffd47d332d547c60de81b76454
            Show
            Gerrit Gerrit added a comment - Change Ic9f378cac19b68ffd47d332d547c60de81b76454, patchset 2 https://gerrit.sio2project.mimuw.edu.pl/1806 SIO-1284 filetracker-server support for logging to stderr Change-Id: Ic9f378cac19b68ffd47d332d547c60de81b76454
            Marek Bardoński made changes -
            Assignee Marek Bardoński [ bdfhjk ] Szymon Acedanski [ accek ]
            Marek Bardoński made changes -
            Status In Progress [ 3 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            Hide
            Permalink
            Szymon Acedański added a comment -
            Well, the change was not correct.
            The logging was redirected to the 'stderr' file in the current directory.
            See abspath in https://github.com/sio2project/filetracker/blob/master/filetracker/servers/run.py#L91
            Show
            Szymon Acedański added a comment - Well, the change was not correct. The logging was redirected to the 'stderr' file in the current directory. See abspath in https://github.com/sio2project/filetracker/blob/master/filetracker/servers/run.py#L91
            Szymon Acedański made changes -
            Resolution Fixed [ 1 ]
            Status Resolved [ 5 ] Reopened [ 4 ]
            Szymon Acedański made changes -
            Assignee Szymon Acedański [ accek ] Marek Bardoński [ bdfhjk ]
            Hide
            Permalink
            Marek Bardoński added a comment -
            which distro of linux are you using, and how the logging event was invoked?
            Show
            Marek Bardoński added a comment - which distro of linux are you using, and how the logging event was invoked?
            Hide
            Permalink
            Szymon Acedański added a comment -
            It's distro-independent. Look at the linked line. It converts "stderr" to an absolute path and passes it to mod_access.
            Show
            Szymon Acedański added a comment - It's distro-independent. Look at the linked line. It converts "stderr" to an absolute path and passes it to mod_access.
            Hide
            Permalink
            Marek Bardoński added a comment -
            Ah... Okey, I see. Sorry about that, going to fix it soon (today I'm too tired after weekend outside city)
            Show
            Marek Bardoński added a comment - Ah... Okey, I see. Sorry about that, going to fix it soon (today I'm too tired after weekend outside city)
            Marek Bardoński made changes -
            Status Reopened [ 4 ] In Progress [ 3 ]
            Hide
            Permalink
            Marek Bardoński added a comment -
            Ok, done. Pull request send for review via github.

            One important thing, I can't test it fully, because it's very hard for me to generate any lighthttpd logging event. Please check on your already set environment if it's working as should (or tell me how you tested it last saturday)
            Show
            Marek Bardoński added a comment - Ok, done. Pull request send for review via github. One important thing, I can't test it fully, because it's very hard for me to generate any lighthttpd logging event. Please check on your already set environment if it's working as should (or tell me how you tested it last saturday)
            Szymon Acedański made changes -
            Sprint The First Sprint [ 2 ] The First Sprint, The Second Sprint [ 2, 3 ]
            Szymon Acedański made changes -
            Rank Ranked higher
            Hide
            Permalink
            Marek Bardoński added a comment -
            After installing locally a lighttpd and enabling filetracker, finally I got reliable verification of my change.

            Now, I've just disabled passing the log file to httpd, and according to *updated* documentation, the default fd to log is stderr.

            This was verified by running the server with specifed local lighttpd binary. No strange file was created on local filesystem, got a rotated log file filetracker-supervisord.log with information

            2013-11-04 12:25:33: (log.c.166) server started

            and filetracker-server-stderr---oioioi-supervisor-QsjitJ.log
             
            Everything look fine, sending change to gerrit.
            Show
            Marek Bardoński added a comment - After installing locally a lighttpd and enabling filetracker, finally I got reliable verification of my change. Now, I've just disabled passing the log file to httpd, and according to *updated* documentation, the default fd to log is stderr. This was verified by running the server with specifed local lighttpd binary. No strange file was created on local filesystem, got a rotated log file filetracker-supervisord.log with information 2013-11-04 12:25:33: (log.c.166) server started and filetracker-server-stderr---oioioi-supervisor-QsjitJ.log   Everything look fine, sending change to gerrit.
            Hide
            Permalink
            Gerrit Gerrit added a comment -
            Change I146f47ff1e347a8c92bf1ff530e2bac1a4f3e5f7, patchset 1
            https://gerrit.sio2project.mimuw.edu.pl/1839

            SIO-1284 filetracker-server support for logging to stderr #2

            Change-Id: I146f47ff1e347a8c92bf1ff530e2bac1a4f3e5f7
            Show
            Gerrit Gerrit added a comment - Change I146f47ff1e347a8c92bf1ff530e2bac1a4f3e5f7, patchset 1 https://gerrit.sio2project.mimuw.edu.pl/1839 SIO-1284 filetracker-server support for logging to stderr #2 Change-Id: I146f47ff1e347a8c92bf1ff530e2bac1a4f3e5f7
            Hide
            Permalink
            Gerrit Gerrit added a comment -
            Change I146f47ff1e347a8c92bf1ff530e2bac1a4f3e5f7, patchset 2
            https://gerrit.sio2project.mimuw.edu.pl/1839

            SIO-1284 filetracker-server support for logging to stderr, second try

            Change-Id: I146f47ff1e347a8c92bf1ff530e2bac1a4f3e5f7
            Show
            Gerrit Gerrit added a comment - Change I146f47ff1e347a8c92bf1ff530e2bac1a4f3e5f7, patchset 2 https://gerrit.sio2project.mimuw.edu.pl/1839 SIO-1284 filetracker-server support for logging to stderr, second try Change-Id: I146f47ff1e347a8c92bf1ff530e2bac1a4f3e5f7
            Hide
            Permalink
            Szymon Acedański added a comment -
            No, we want mod_accesslog to log to stderr.
            Show
            Szymon Acedański added a comment - No, we want mod_accesslog to log to stderr.
            Hide
            Permalink
            Marek Bardoński added a comment -
            We want a switch to enable/disable mod_accesslog or just leave mod_accesslog with stderr, when log file isn't specified?
            Show
            Marek Bardoński added a comment - We want a switch to enable/disable mod_accesslog or just leave mod_accesslog with stderr, when log file isn't specified?
            Hide
            Permalink
            Marek Bardoński added a comment -
            that mean, my first idea now is to enable mod_accesslog in filetracker as constant, and if user specify log file in supervisord.conf, then logs go to it, if not, then logs go to stderr. Right?
            Show
            Marek Bardoński added a comment - that mean, my first idea now is to enable mod_accesslog in filetracker as constant, and if user specify log file in supervisord.conf, then logs go to it, if not, then logs go to stderr. Right?
            Hide
            Permalink
            Marek Bardoński added a comment -
            I've prepared a pull request to visualize my idea.

            Anyway, still when I'll enter http://students.mimuw.edu.pl:9999/cokolwiek, then filetracker server respond with error 400, but no information about this is generated in logs. Please confirm, that this is a bad sign, and I'll investigate more.

            Show
            Marek Bardoński added a comment - I've prepared a pull request to visualize my idea. Anyway, still when I'll enter http://students.mimuw.edu.pl:9999/cokolwiek, then filetracker server respond with error 400, but no information about this is generated in logs. Please confirm, that this is a bad sign, and I'll investigate more.
            Hide
            Permalink
            Szymon Acedański added a comment -
            Yes, that's a bad sign. Before this change such a request was logged (albeit not to stderr, but to file named 'stderr').

            Have a look at what was logged before your first change, and the intended behavior is to log the same thing, just through stderr, so that supervisord will handle it. I think it's clear. Please do not send changes for review until you actually verified that this works.
            Show
            Szymon Acedański added a comment - Yes, that's a bad sign. Before this change such a request was logged (albeit not to stderr, but to file named 'stderr'). Have a look at what was logged before your first change, and the intended behavior is to log the same thing, just through stderr, so that supervisord will handle it. I think it's clear. Please do not send changes for review until you actually verified that this works.
            Hide
            Permalink
            Gerrit Gerrit added a comment -
            Change I146f47ff1e347a8c92bf1ff530e2bac1a4f3e5f7, patchset 3
            https://gerrit.sio2project.mimuw.edu.pl/1839

            SIO-1284 filetracker-server support for logging to stderr (fix to last commit)

            Change-Id: I146f47ff1e347a8c92bf1ff530e2bac1a4f3e5f7
            Show
            Gerrit Gerrit added a comment - Change I146f47ff1e347a8c92bf1ff530e2bac1a4f3e5f7, patchset 3 https://gerrit.sio2project.mimuw.edu.pl/1839 SIO-1284 filetracker-server support for logging to stderr (fix to last commit) Change-Id: I146f47ff1e347a8c92bf1ff530e2bac1a4f3e5f7
            Hide
            Permalink
            Marek Bardoński added a comment -
            Okey, now it's verified. I got this kind of messages processed via stderr to /logs/filetracker-supervisord.log

            Example message "77.115.72.123 students.mimuw.edu.pl:9999 - [05/Nov/2013:12:14:56 +0100] "GET /cokolwiekk HTTP/1.1" 404 345 "-" "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.123.1599.101 Safari/537.36"
            "

            No change to filetracker itself is need.
            Show
            Marek Bardoński added a comment - Okey, now it's verified. I got this kind of messages processed via stderr to /logs/filetracker-supervisord.log Example message "77.115.72.123 students.mimuw.edu.pl:9999 - [05/Nov/2013:12:14:56 +0100] "GET /cokolwiekk HTTP/1.1" 404 345 "-" "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.123.1599.101 Safari/537.36" " No change to filetracker itself is need.
            Hide
            Permalink
            Gerrit Gerrit added a comment -
            Change I146f47ff1e347a8c92bf1ff530e2bac1a4f3e5f7, patchset 4
            https://gerrit.sio2project.mimuw.edu.pl/1839

            SIO-1284 filetracker-server support for logging to stderr (fix to last commit)

            Change-Id: I146f47ff1e347a8c92bf1ff530e2bac1a4f3e5f7
            Show
            Gerrit Gerrit added a comment - Change I146f47ff1e347a8c92bf1ff530e2bac1a4f3e5f7, patchset 4 https://gerrit.sio2project.mimuw.edu.pl/1839 SIO-1284 filetracker-server support for logging to stderr (fix to last commit) Change-Id: I146f47ff1e347a8c92bf1ff530e2bac1a4f3e5f7
            Marek Bardoński made changes -
            Status In Progress [ 3 ] Resolved [ 5 ]
            Assignee Marek Bardoński [ bdfhjk ] Szymon Acedański [ accek ]
            Resolution Fixed [ 1 ]
            Szymon Acedański made changes -
            Status Resolved [ 5 ] Closed [ 6 ]
            Szymon Acedański made changes -
            Fix Version/s The First Sprint [ 11500 ]
            Transition Time In Source Status Execution Times Last Executer Last Execution Date
            New New Open Open
            		59d 6h 24m 1 Szymon Acedański 2013-10-5 18:18
            Open Open In Progress In Progress
            		9d 15h 30m 1 Marek Bardoński 2013-10-15 9:49
            Resolved Resolved Reopened Reopened
            		4d 6h 46m 1 Szymon Acedański 2013-10-26 23:29
            Reopened Reopened In Progress In Progress
            		1d 11h 27m 1 Marek Bardoński 2013-10-28 9:56
            In Progress In Progress Resolved Resolved
            		17d 13h 29m 2 Marek Bardoński 2013-11-7 16:32
            Resolved Resolved Closed Closed
            		326d 4h 9m 1 Szymon Acedański 2014-09-29 21:41

              People

              • Assignee:
                Szymon Acedański
                Reporter:
                Szymon Acedański
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Agile