Jonkman Microblog
  • Login
Show Navigation
  • Public

    • Public
    • Network
    • Groups
    • Popular
    • People

Notices by musicman (musicman@nu.federati.net), page 99

  1. musicman (musicman@nu.federati.net)'s status on Wednesday, 06-Mar-2019 14:19:34 EST musicman musicman
    in reply to
    • musicman
    interesting: https://news.ycombinator.com/item?id=12623249
    In conversation Wednesday, 06-Mar-2019 14:19:34 EST from nu.federati.net permalink
  2. musicman (musicman@nu.federati.net)'s status on Wednesday, 06-Mar-2019 14:17:37 EST musicman musicman
    ok, I've heard of dnotify and inotify, but now "Later, fanotify was created to overcome this issue"

    is fanotify before or after inotify?
    In conversation Wednesday, 06-Mar-2019 14:17:37 EST from nu.federati.net permalink
  3. musicman (musicman@nu.federati.net)'s status on Wednesday, 06-Mar-2019 11:38:25 EST musicman musicman
    in reply to
    • musicman
    ok, digging a bit more:

    "The strace file should have been run with 'strace -T -ttt [PROGRAM]"
    In conversation Wednesday, 06-Mar-2019 11:38:25 EST from nu.federati.net permalink
  4. musicman (musicman@nu.federati.net)'s status on Wednesday, 06-Mar-2019 11:32:25 EST musicman musicman
    Anybody ever used https://clusterbuffer.wordpress.com/strace-analyzer/ ? It doesn't seem to work !sysadmin
    In conversation Wednesday, 06-Mar-2019 11:32:25 EST from nu.federati.net permalink

    Attachments

    1. File without filename could not get a thumbnail source.
      Strace Analyzer
      By clusterbuffer from clusterbuffer

      Introduction

      I’ve written a number of articles around the web on the subject of using strace to get information about the IO in your application. But the hard thing is how to take seemingly endless lines in the output file and extract useful information. I learned at Panasas that you can take that strace data and extract useful IO information. For example you can extract a great deal of statistical information such as how much time was spent on IO, how much data was read, how much data was written, the IO rates, what files were opened, how many lseek’s or llseek’s were used, etc. So I wrote my own strace analyzer to take strace output files and produce something that is reasonably useful.

      In the HPC world many of the applications use MPI which means that you get an individual strace file for each MPI process. But many times we want to produce an “application-level” view of the strace. So there is a new tool to take the summary of the strace for each process and combine them to give you the total view of things.

      When I started the project I used it as an excuse for learning Perl. Perl has some great tools for this type of thing but this was the only application where I used Perl. So I decided to rewrite everything in Python, which I know better. Plus Python has some tools that I want to use for longer term development (GUI, plotting, etc.).

      The Python version of the strace analyzer that uses the elapsed time capability of the strace function. You should consider this to be the current version of the strace analyzer. It has plotting capability via matplotlib and can produce an uncompressed pickle output of the major data structures that will be used with additional tools.

      The analyzer is released under GPL v2. All the supporting tools are also released under GPL v2 as well.

      Getting Started


      To get started with the strace analyzer and other tools, you first need to create an strace file of your application. Then you can process it through the tools. So let’s start there – how to create an strace file.

      How to create an strace file

      Creating the strace file that is appropriate for the strace analyzer involves just a few options. The required options are:

      strace -T -ttt -o strace.out <application>

      The “-T” option cause strace to print out the elapsed time at the end of the line in the form, <0.000197>. The option “-ttt” causes microsecond timing but does so in seconds since the epoch (makes life easier for applications that run a long time and for MPI applications). The “-o” option specifies the output file. In this case the output file is called “strace.out” but it can be called anything you like.

      If you want to reduce the amount of output then you can try the following:

      strace -T -ttt -e trace=open,close,read,write,lseek,llseek,_llseek,fsync,fstat,fstat64,... -o strace.out

      which specifies the specific functions you want strace to track. You can add almost any IO function you want to the “trace=…” line. The options that the strace analyzer current recognizes are,

      • read
      • write
      • open
      • close
      • lseek
      • llseek
      • __llseek
      • stat
      • stat64
      • fstat
      • chmod
      • access
      • rename
      • mkdir
      • getdents
      • fcntl
      • unlink
      • fseek
      • rewind
      • ftell
      • fgetpos
      • fsetpos
      • fclose
      • fsync
      • creat
      • readdir
      • opendir
      • rewinddir
      • scandir
      • seekdir
      • telldir
      • flock
      • lockf

      Other IO operations will be added later (asynchronous IO, fread, fwrite, etc.).

      Simple Example:


      To get a feel for how things work, let’s run a simple example. Since I’m on the older side, I will write a simple Fortran program that opens a file, writes to it, and closes the file. The Fortran code is below:
      Fortran code goes here.
      

      Now that you have an strace file let’s run it through the strace analyzer.

      Strace_analyzer Options:

      There are several options for the strace analyzer. You an get a listing of the options by using the help option. For example,

      $ ./strace_analyzer_ng_0.03.pl -help
      Usage: strace-analyze [OPTION]… [FILE]
      Analyzes strace output for IO functions. It creates statistics
      on IO functions and performance of the read and write functions.
      The strace file should have been run with ‘strace -tt [PROGRAM]

      -d Debug – echo the input lines to stdout and provide debugging information
      -debug same as -d
      –d same as -d
      –debug same as -d
      -h Produces help output (this message)
      -help same as -h
      –h same as -h
      –help same as -h
      -f [FILE] input a list of file sizes for examining IO functions
      The format of the file is:
      1024
      8192
      …
      32768
      1048576
      -file same as -f
      –f same as -f
      –file same as -f
      -dat Produce .dat files for plotting
      –dat same as -dat
      -s Create simulator code – file is simulator.c
      -sim same as -s
      –s same as -s
      –sim same as -s
      -short Short version of output
      –short same as -short

      The normal options that I use are “-short -dat”. It produces much less output than usual and also produces data files fit for plotting using something like Grace.

      Example output:

      The output from the strace analyzer can either be very long or relatively short. It also produces some comma delimited files (csv) as well as the data files that can be used for plotting.

      By default you will get two files: strace.log and whatever file you pipe the output to (stdout). So for example,

      strace -short -dat < file.strace > strace.out

      will produce strace.log and strace.out files. Here is a quick example of the strace.out file:


      Analysis Output
      ===============

      Number of Lines in strace file: 7072

      —————-
      — Time Stats —
      —————-
      Elapsed Time for run: 0.526419 (secs)
      Total IO Time: 0.121526 (secs)
      Total IO Time Counter: 7025
      Percentage of Total Time = 23.085406%

      ———————-
      — IO Command Count —
      ———————-
      Command Count
      ==============================
      stat64 1
      open 6
      close 7
      access 6
      lseek 4000
      read 4
      write 2998

      ———————-
      — Write Statistics —
      ———————-

      — File sizes for write() function —

      IO Size Range Number of Functions
      =======================================================
      ( 1) 0KB < < 1KB 1998
      ( 2) 1KB < < 8KB 0
      ( 3) 8KB < < 32KB 1000
      ( 4) 32KB < < 128KB 0
      ( 5) 128KB < < 256KB
      0
      ( 6) 256KB < < 512KB
      0
      ( 7) 512KB < < 1000KB
      0
      ( 8) 1000KB < < 10MB
      0
      ( 9) 10MB < < 100MB
      0
      ( 10) 100MB < < 1GB
      0
      ( 11) 1GB < < 10GB
      0
      ( 12) 10GB < < 100GB
      0
      ( 13) 100GB < < 1TB
      0
      ( 14) 1TB < < 10TB
      0

      — WRITE SUMMARY —
      Total number of Bytes written = 8,011,996 (8.011996 MB)
      Number of Write function calls = 2,998

      Average (mean) Bytes per call = 2,672.4469646431 (bytes) (0.0026724469646431 MB)
      Standard Deviation: 3,771.86653967916 (bytes) (0.00377186653967916 MB)
      Mean Absolute Deviation: 3,922.7267533689 (bytes) (0.0039227267533689 MB)
      Median Bytes per call = 4 (bytes) (4e-06 MB)
      Median Absolute Deviation: 2,668.4469646431 (bytes) (0.0026684469646431 MB)

      Time for slowest write function (secs) = 0.000439
      Line location in file: 4777

      ———————
      — Read Statistics —
      ———————

      — File sizes for read function() —

      IO Size Range Number of Functions
      =======================================================
      ( 1) 0KB < < 1KB 2002
      ( 2) 1KB < < 8KB 0
      ( 3) 8KB < < 32KB 1000
      ( 4) 32KB < < 128KB0
      ( 5) 128KB < < 256KB
      0
      ( 6) 256KB < < 512KB
      0
      ( 7) 512KB < < 1000KB
      0
      ( 8) 1000KB < < 10MB
      0
      ( 9) 10MB < < 100MB
      0
      ( 10) 100MB < < 1GB
      0
      ( 11) 1GB < < 10GB
      0
      ( 12) 10GB < < 100GB
      0
      ( 13) 100GB < < 1TB
      0
      ( 14) 1TB < < 10TB
      0

      — READ SUMMARY —
      Total number of Bytes read = 2,048 (0.002048 MB)
      Number of Read function calls = 4

      Average (mean) Bytes per call = 512 (bytes) (0.000512 MB)
      Standard Deviation: 0 (bytes) (0 MB)
      Mean Absolute Deviation: 512 (bytes) (0.000512 MB)
      Median Bytes per call = 512 (bytes) (0.000512 MB)
      Median Absolute Deviation: 0 (bytes) (0 MB)

      Time for slowest read function (secs) = 0.000439
      Line location in file: 4777

      ———————-
      — Close Statistics —
      ———————-

      — CLOSE SUMMARY —
      Total number of close function calls = 7
      Average time for close function calls (secs) = 0.000010
      Maximum Time for close function (secs) = 0.000015
      Line location in file: 7069

      ———————
      — Open Statistics —
      ———————

      — OPEN SUMMARY —
      Total number of open function calls = 6
      Average time for open function calls (secs) = 0.000000
      Maximum Time for open function (secs) = 0.000042
      Line location in file: 66

      ————————————
      — lseek unit activity Statistics —
      ————————————

      unit Number of lseeks
      ========================
      3 4000

      ———————
      — IOPS Statistics —
      ———————

      Maximum Write IOPS = 2998 occured at 1 seconds
      Write IOPS = 2998

      Maximum Read IOPS = 4 occured at 1 seconds
      Read IOPS = 4

      Maximum Total IOPS = 7021 occured at 1 seconds
      Total IOPS = 3512

      *****************
      Final IOPS report
      Write IOPS = 2998
      Read IOPS = 4
      Total IOPS = 3512
      *****************

      — File Statistics (per file basis) —
      Filename Read Bytes Bytes/sec Write Bytes Bytes/sec
      =========================================================================================================================================================================
      junk.out 2,048 0 8,011,996 0
      DONE

      Here are example of what you could expect to see if you plot the data files. The first plot is of Read IOPS vs. Time for the simple example (really simple). It is the IOPS for all files although the analyzer will produce data files for each file opened.

  5. musicman (musicman@nu.federati.net)'s status on Wednesday, 06-Mar-2019 10:43:13 EST musicman musicman
    • Black Metal
    sounds a lot like if Metallica started incorporating some !blackmetal

    https://wackenmetalbattlecanada.bandcamp.com/track/tymo-killer-krom

    #ccmusic
    In conversation Wednesday, 06-Mar-2019 10:43:13 EST from nu.federati.net permalink

    Attachments

    1. Invalid filename.
      Tymo - Killer Krom, by Wacken Metal Battle Canada
      from Wacken Metal Battle Canada
      from the album Wacken Metal Battle Canada Compilation Vol. 5
  6. musicman (musicman@nu.federati.net)'s status on Wednesday, 06-Mar-2019 10:15:21 EST musicman musicman
    I wouln't suggest this is legal, but it's interesting and I guess no one has complained: https://nu.federati.net/url/209386
    In conversation Wednesday, 06-Mar-2019 10:15:21 EST from nu.federati.net permalink

    Attachments

    1. Invalid filename.
      08 - You & Me (Featuring Ryan Flow) - Broad Street Boogie - Releases - blocSonic - A Maine based netlabel & music label featuring hip-hop to pop to indie to rock and everything in between
      A netlabel / record label based in the State of Maine that focuses on delivering only the best new, eclectic music and distributing it with a professional sheen that’s rarely seen coming from independent music labels.
  7. musicman (musicman@nu.federati.net)'s status on Tuesday, 05-Mar-2019 17:25:45 EST musicman musicman
    Is there a tool that will look at an strace and tell you if all the file handles got closed? #Linux
    In conversation Tuesday, 05-Mar-2019 17:25:45 EST from nu.federati.net permalink
  8. musicman (musicman@nu.federati.net)'s status on Tuesday, 05-Mar-2019 17:19:11 EST musicman musicman
    Every time I see O_RDONLY I think https://weheartit.com/entry/368875
    In conversation Tuesday, 05-Mar-2019 17:19:11 EST from nu.federati.net permalink

    Attachments

    1. Invalid filename.
      demotivational posters - O RLY - lolowl lolcats, owl, birds, lolowl, harry potter, rly 194
      from We Heart It
      Image discovered by acidkiss. Find images and videos about owl, oh really and Hedwige on We Heart It - the app to get lost in what you love.
  9. Michael Downey 🚩 (downey@floss.social)'s status on Friday, 01-Mar-2019 13:14:59 EST Michael Downey 🚩 Michael Downey 🚩

    So the Open Source Initiative election is coming up very soon:

    https://opensource.org/elections

    I'm happy to see such a richly diverse set of candidates this election. #OpenSource is about freedom *for all*, so let's work together to make sure we aren't left with an all-white male board from only the US and EU.

    The entire world needs strong community stewardship of Open Source more now than ever, and we are stronger when we stand together.

    #FreeSoftware #FLOSS #FOSS

    In conversation Friday, 01-Mar-2019 13:14:59 EST from floss.social permalink Repeated by musicman
  10. musicman (musicman@nu.federati.net)'s status on Tuesday, 05-Mar-2019 16:56:53 EST musicman musicman
    Stop using strace and call your doctor at once if you have:

    problems with your vision or hearing;
    hallucinations, (see or hearing things that are not real), thoughts about suicide or hurting yourself;
    depressed mood, crying spells, changes in behavior, feeling angry or irritable;
    loss of interest in things you enjoyed before, feeling hopeless or guilty;
    sleep problems, extreme tiredness, trouble concentrating;
    changes in weight or appetite;
    a seizure (convulsions), sudden numbness or weakness;
    muscle weakness, pain in your bones or joints or in your back;
    severe diarrhea, rectal bleeding, bloody or tarry stools;
    pale skin, feeling light-headed or short of breath;
    severe stomach or chest pain, pain when swallowing; or
    dark urine, or jaundice (yellowing of your skin or eyes).
    In conversation Tuesday, 05-Mar-2019 16:56:53 EST from nu.federati.net permalink
  11. musicman (musicman@nu.federati.net)'s status on Tuesday, 05-Mar-2019 16:04:06 EST musicman musicman
    in reply to
    • musicman
    "The -v (verbose) switch tells strace to print the more information about environment for many calls. This is very noticeable on the “stat” call. For instance, the first line of a strace without the verbose option looks like this:
    1489 execve("./open-silent.pl", ["./open-silent.pl"], [/* 31 vars */]) = 0
    with the verbose option, I see:
    execve("./open-silent.pl", ["./open-silent.pl"], ["HOSTNAME=nt12199.product.cpanel."..., "TERM=xterm-256color", "SHELL=/bin/bash", "HISTSIZE=1000", "SSH_CLIENT=10.1.7.152 56078 22", "PERL5LIB=/home/mary/perl5/lib/pe"..., "OLDPWD=/home/mary", "PERL_MB_OPT=--install_base "/hom"..., "SSH_TTY=/dev/pts/0", "USER=mary", "LS_COLORS=rs=0:di=01;34:ln=01;36"..., "MAIL=/var/spool/mail/mary", "PATH=/usr/local/jdk/bin:/home/ma"..., "PWD=/home/mary/public_html", "JAVA_HOME=/usr/local/jdk", "EDITOR=pico", "LANG=en_US.UTF-8", "HISTCONTROL=ignoredups", "SHLVL=1", "HOME=/home/mary", "LS_OPTIONS=--color=tty -F -a -b "..., "PERL_LOCAL_LIB_ROOT=/home/mary/p"..., "LOGNAME=mary", "VISUAL=pico", "CVS_RSH=ssh", "CLASSPATH=.:/usr/local/jdk/lib/c"..., "SSH_CONNECTION=10.1.7.152 56078 "..., "LESSOPEN=|/usr/bin/lesspipe.sh %"..., "PERL_MM_OPT=INSTALL_BASE=/home/m"..., "G_BROKEN_FILENAMES=1", "_=/usr/bin/strace"]) = 0"

    Well, yes, that is quite a bit more verbose!
    In conversation Tuesday, 05-Mar-2019 16:04:06 EST from nu.federati.net permalink
  12. musicman (musicman@nu.federati.net)'s status on Tuesday, 05-Mar-2019 15:33:42 EST musicman musicman
    in reply to
    • musicman
    I by work, I mean https://blog.cpanel.com/starting-with-strace/
    In conversation Tuesday, 05-Mar-2019 15:33:42 EST from nu.federati.net permalink

    Attachments

    1. File without filename could not get a thumbnail source.
      Starting with Strace
      By Mary from cPanel Blog
      Starting with Strace
  13. musicman (musicman@nu.federati.net)'s status on Tuesday, 05-Mar-2019 15:27:36 EST musicman musicman
    knock on wood, but all of my cases are set to On Customer. I do have some work to do on a couple of them though.
    In conversation Tuesday, 05-Mar-2019 15:27:36 EST from nu.federati.net permalink
  14. musicman (musicman@nu.federati.net)'s status on Tuesday, 05-Mar-2019 12:30:32 EST musicman musicman
    one of my favorite tracks on the label: https://nu.federati.net/url/209178 #ccmusic #piano #hiphop
    In conversation Tuesday, 05-Mar-2019 12:30:32 EST from nu.federati.net permalink

    Attachments

    1. Invalid filename.
      16 - The GetBack (Dr. Mindflip's Medicinal Mix) - Everything is Different: Impossebulls Remixxed - Releases - blocSonic - A Maine based netlabel & music label featuring hip-hop to pop to indie to rock and everything in between
      A netlabel / record label based in the State of Maine that focuses on delivering only the best new, eclectic music and distributing it with a professional sheen that’s rarely seen coming from independent music labels.
  15. musicman (musicman@nu.federati.net)'s status on Monday, 04-Mar-2019 17:59:15 EST musicman musicman
    Any good resources for gdb?
    In conversation Monday, 04-Mar-2019 17:59:15 EST from nu.federati.net permalink
  16. musicman (musicman@nu.federati.net)'s status on Monday, 04-Mar-2019 17:13:56 EST musicman musicman
    some sites are calling for 15 inches of snow.

    some sites are not calling for any snow.

    The storm is currently over Japan, so I guess we'll see...
    In conversation Monday, 04-Mar-2019 17:13:56 EST from nu.federati.net permalink
  17. musicman (musicman@nu.federati.net)'s status on Monday, 04-Mar-2019 16:46:20 EST musicman musicman
    • Version Control Software
    My understanding is we are getting close on this one, so if you are interested, let me know ASAP:


    Position Title: Technical Support Engineer
    Location: #Sydney, #Australia, #NSW
    Reports to: Director of Support (aka my boss)



    Apply to Perforce today if this sounds interesting to you! We’re a leading global software company looking for smart, fun, talented team members. At Perforce, you’ll enjoy competitive benefits while working with and learning from some of the best and brightest in business. Before you know it, you’ll be in the middle of a rewarding career at a company headed in one direction: upward.

    Position Summary
    Perforce is seeking a Tier 2, Technical Support Engineer to join our Global Support Team, liaising with members from Sales, Professional Services and, Development to assist in resolving a wide variety of customer issues. You would be responsible for ensuring the success of our customers by effectively providing dependable and timely resolution to Perforce's Helix Core software. The ideal candidate is expected to be self-motivated, proactive, results-oriented and able to provide a high level of customer satisfaction through the delivery of world-class technical support services.

    Essential Functions

    Represent Perforce as the first point of contact for their technical support queries.
    Review scope of customer issue and determine best course for resolution.
    Develop and maintain technical expertise in assigned areas of product functionality and utilize it effectively to help customers.
    Resolve customer issues expeditiously.
    Resolve database and performance issues.
    Research, document, and escalate cases according to procedure.
    Customer driven feedback to functional areas in order to influence process/product improvements.
    Author technical documents on common issues and solutions in order to build the knowledge base.
    Positive attitude - Support engineers are required to be respectful, fair, gracious, and knowledgeable.
    Create and set up test environments.

    Required Education, Experience and Skills

    2 or more years’ experience providing support directly to enterprise customers.
    Technical or Bachelor’s Degree in IT, CS or similar.
    #Linux experience
    Basic networking experience
    Outstanding customer service skills.
    Strong analytics and problem solving skills.
    Ability to work in a team.
    Excellent written and verbal communication skills.
    Able to work well under pressure and prioritize accordingly.
    Organized and dedicated.
    Good attention to detail.
    Experience with !Perforce, #Git, or other version control software is desirable
    In conversation Monday, 04-Mar-2019 16:46:20 EST from nu.federati.net permalink
  18. musicman (musicman@nu.federati.net)'s status on Monday, 04-Mar-2019 15:26:55 EST musicman musicman
    Someone just took this ticket. I hope their response is "wtf?"
    In conversation Monday, 04-Mar-2019 15:26:55 EST from nu.federati.net permalink
  19. musicman (musicman@nu.federati.net)'s status on Monday, 04-Mar-2019 14:14:38 EST musicman musicman
    • Craig Maloney ☕
    @craigmaloney this one sticks out from the compilation if you haven't made it through: https://wackenmetalbattlecanada.bandcamp.com/track/sovereign-council-the-human-condition
    In conversation Monday, 04-Mar-2019 14:14:38 EST from nu.federati.net permalink

    Attachments

    1. Invalid filename.
      Sovereign Council - The Human Condition, by Wacken Metal Battle Canada
      from Wacken Metal Battle Canada
      from the album Wacken Metal Battle Canada Compilation Vol. 5
  20. musicman (musicman@nu.federati.net)'s status on Monday, 04-Mar-2019 11:00:07 EST musicman musicman
    Does auditd use inotify? !linux
    In conversation Monday, 04-Mar-2019 11:00:07 EST from nu.federati.net permalink
  • After
  • Before
  • Help
  • About
  • FAQ
  • TOS
  • Privacy
  • Source
  • Version
  • Contact

Jonkman Microblog is a social network, courtesy of SOBAC Microcomputer Services. It runs on GNU social, version 1.2.0-beta5, available under the GNU Affero General Public License.

Creative Commons Attribution 3.0 All Jonkman Microblog content and data are available under the Creative Commons Attribution 3.0 license.

Switch to desktop site layout.