I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.
-
Upload
samuel-fletcher -
Category
Documents
-
view
215 -
download
0
Transcript of I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.
![Page 1: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/1.jpg)
I Hate Cron JobsJ.A. Hourclé - 2015 DC-Baltimore Perl
Workshop
![Page 2: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/2.jpg)
I <3 cron
![Page 3: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/3.jpg)
![Page 4: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/4.jpg)
![Page 5: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/5.jpg)
I </3 cronjobs
![Page 6: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/6.jpg)
![Page 7: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/7.jpg)
![Page 8: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/8.jpg)
![Page 9: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/9.jpg)
![Page 10: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/10.jpg)
Problems:•I have no idea which of these
messages are important.•Routine output looks the exact
same as a critical error•You start to ignore those cronjobs
(like clamAV) that generate messages every day.
![Page 11: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/11.jpg)
Solution:
•If the cron job doesn’t generate output, it won’t trigger an e-mail.
![Page 12: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/12.jpg)
Problem #2:•This assumes that it sends to
STDERR when something goes wrong.
•And that there’s enough context in STDERR to debug the problem
•You’ve just trashed whatever other output may have been useful
![Page 13: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/13.jpg)
Solution #2
•Capture output, only display on error
![Page 14: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/14.jpg)
Solution #2.1
•Capture output, only display on error
![Page 15: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/15.jpg)
Problems #3-5:•Programs that only write to
STDOUT•When the absense of output is the
error.•Trying to find the actual error in
really spammy commands• like rsync
![Page 16: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/16.jpg)
Solution #3:
![Page 17: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/17.jpg)
Solution #3:
![Page 18: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/18.jpg)
Problem #6
•The ‘stuck’ cronjob:• Every hour, for the last week, cron jobs
have been starting but not completing.
![Page 19: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/19.jpg)
Problem #n
•(insert problem here)
![Page 20: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/20.jpg)
![Page 21: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/21.jpg)
Use Case #1•Only report if output on STDERR•Otherwise, supress STDOUT•(default)• disable with --ignore-stderr and
--ignore-stdout=0 or --verbose• or use --merge
![Page 22: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/22.jpg)
Use Case #2
•Only report if non-zero status•(default)• --ignore-return to disable reporting of
non-zero status
![Page 23: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/23.jpg)
![Page 24: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/24.jpg)
![Page 25: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/25.jpg)
Use Case #3:
•Report if a process runs too long•--timer=(seconds)
•--timeout=(seconds)
![Page 26: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/26.jpg)
![Page 27: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/27.jpg)
![Page 28: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/28.jpg)
![Page 29: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/29.jpg)
![Page 30: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/30.jpg)
Use Case #4•Prevent new jobs if an older job is
still running•--once• --ignore-once if you don’t want this to
automatically be reported as an error• --name to set the lockfile name
![Page 31: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/31.jpg)
![Page 32: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/32.jpg)
![Page 33: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/33.jpg)
Use Case #5
•Kill old instances if still running before starting the new job
• --once=2
![Page 34: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/34.jpg)
Use Case #6:
•Debugging a cron job• Dump %ENV before running
•--verbose=2 : always generate a message
•--verbose=3 : dumps %ENV
![Page 35: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/35.jpg)
![Page 36: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/36.jpg)
Use Case #7
•Highlight what the error is•--lines=(regex) for ‘known good’
lines.
![Page 37: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/37.jpg)
![Page 38: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/38.jpg)
![Page 39: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/39.jpg)
![Page 40: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/40.jpg)
![Page 41: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/41.jpg)
Use Case #8
•Don’t report if the expected output•--lines=(regex)• optional and repetitive items
•--template=(regex)• must be present
![Page 42: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/42.jpg)
![Page 43: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/43.jpg)
![Page 44: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/44.jpg)
Config file support
•--file=(path_to_YAML)
•any command line args override options set in the file
![Page 45: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/45.jpg)
![Page 46: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/46.jpg)
![Page 47: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/47.jpg)
Doesn’t doeverything
![Page 48: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/48.jpg)
Doesn’t:
•Report before the program ends
![Page 49: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/49.jpg)
Doesn’t:•Generate its own e-mail• So we could change the subject line on
success or failure• see App::CronJob
• or change the recipient list based on sucess/fail,or specific error states.
![Page 50: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/50.jpg)
Doesn’t:
•Track success/failrate• only report (x) failures in a row• or (y) failures per (time)
![Page 51: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/51.jpg)
Doesn’t:
•Limit resources• see Process::Govern
![Page 52: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/52.jpg)
Helper script
![Page 53: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/53.jpg)
Problem:Not released
![Page 54: I Hate Cron Jobs J.A. Hourclé - 2015 DC-Baltimore Perl Workshop.](https://reader036.fdocuments.in/reader036/viewer/2022070605/5a4d1ad27f8b9ab059971e35/html5/thumbnails/54.jpg)
Solar Dynamics Observatory (SDO)Atmospheric Imaging Assembly (AIA)
171Ångstrom ; 2010/07/08 17:45:48UT ; 2x2 binned