YAPC::EU 2016 Day 1

These are my notes from the talks I attended at the first day of YAPC::EU 2016.

CSV headers at your service (Tux)

• usually use a comma to separate columns
• need to know separator before parsing as it could be a semicolon, pipe (or tab), hence determine this from the header
• line endings recommends \r\n in spec; unix: \n, old mac: \r
• detecting encoding is difficult; does one detect binary, utf8, latin1?
  • there is a BOM
  • basically a magic number for the encoding at the start of the csv file, which turns up in the header)
• CSV can also contain cross references to other CSV files
  • multiple headers can appear
  • links between tables via repeated header name
  • need to handle multiple separators, multiple encodings, etc
• showed how to handle these issues in Perl 5 and 6
• the solution lead to a new method -> header(), which handles the separation character, BOM etc.
• can now parse files for a client

BDD web application testing with pherkin and weasel (Erik Huelsmann)

• Erik maintains LedgerSMB (on GitHub: https://github.com/ledgersmb/LedgerSMB/)
• wanted to update a legacy codebase
  • non modular
  • no tests
  • code reuse by cross-requiring scripts
  • regular breakage
  • manual testing required hence breakages went undetected and into production
• solution with hand-written HTTP requests in self written web testing framework -> unmaintainable
• new solution: move to BDD
  • tests read as documentation
  • tests very high level
  • good experience from co-dev
• chose Test::BDD::Cucumber because:
  • active development
  • stand alone cli (pherkin)
  • colour output
  • integration with Test::Builder (can run within prove)
  • tries to be Gherkin compatible
• improved code coverage from 11% to 15% with first tests (how much code in total though?)
• growing pains
  • needed pre- and post- test-run, feature, scenario, and step hooks, since setup and tear-down not available (although Before and After blocks exist, these only exist at the step level)
  • looks like one can hook in much more behaviour via an extension rather than via Begin/After
• copied concept of extensions from behat into pherkin
  • now can hook in own code into pherkin elements
• Weasel (based on Mink from behat)
  • flexibly proxying DOM tree (web application) behaviours to Perl
  • want to solve:
    • hardcoded xpath expressions in Mink
    • repeated locators in code
    • “one class fits all elements”
    • abstract the technology being driven, e.g. Selenium, Sahi, …
  • building blocks
    • session, driver, element
    • don’t hard-code behaviour (as Mink does…)
  • write xpath locators via mnemonic $self->find(".*button", label=>"Ok") instead of complicated xpath expressions
  • xpath expressions are dynamically composed
  • see also Weasel::Widgets::Dojo
• now code coverage up to 25% and increasing (~50k lines of Perl)
• wants to make a website documenting in-browser testing in Perl (and Weasel)

XS Accessors Under the Hood (Sergey Aleynikov)

• detailed look at how to construct accessors (get, set) in Perl
• comparison of runtimes of different implementations
• looks at what code Perl runs internally
• showed initial version in XS; not thread safe, leaks memory; works “most of the time”
• alternative: use PERL_MAGIC in XS
• a reference to the key could exist outside the scope when it’s passed in to the accessor, hence need to copy the key into a string before turning it into a magic variable
• does opcode replacement to avoid work and thus optimise things
• mentioned DBIx::Class example which seemed to save a lot of work
• quite complex material to understand…

A few git bits (Lee Johnson)

• some git tips from his git workflow (https://github.com/leejo/a_few_git_bits)
• git-wtf shows branches one is working on and the state they’re in
• git-tools repo has the timesheet command to see what one has done over a day or week or whatever
• git-extras: many commands to help with workflow

Ref::Util (Sawyer X)

• started from a micro-optimisation idea from Jarkko Hietaniemi
• ref does two things: tells you what kind reference you have, or the name of the package you’re in. This can cause problems due to which thing is being inspected.
• one can use reftype in Scalar::Util to check more reliably; have to combine with ref.
• can take this idea and turn it into a function, e.g. is_arrayref.
• to make the function faster, use XS
• wasn’t that much faster than the pure Perl code
• one could try to use a custom op to perform the same task
• perl -MO=Concise -e '<perl code>' => to see the opcodes Perl creates to run the code
• can rewrite the optree to remove unneeded ops, thus reducing work
• Ref::Util collects the utility functions to check for various kinds of references

Vulnerability discovery, response and remediation for developers (John Lightsey)

• wants to discuss about how vulnerabilities are supposed to be handled
• vulnerability
  • unexpected behaviour in a software system that has a negative security impact
• exploit
  • software or process which uses one or more vulnerabilities to achieve a concrete impact on software system’s confidentiality, integrity or availability
• one wants the lifecycle of vulnerabilities to limit the profitability of exploits being created and used
• can be that many small mistakes (which don’t look like vulnerabilities) can in the sum be turned into (or used in) an exploit.

From npm to ‘Battle tested’ (Oriol Soriano Vila)

• tried to fix an issue in Grafana (add a configurable timeout for graph creation)
• found it was hard to build the package and its dependencies (it’s written in Node.js and hence npm install blew up
• why had he not found this in Perl?
  • -> CPAN testers
• the issue with the removal of left-pad from npm would not happen in Perl since we have BackPAN
• can use Carton to handle dependencies. Creates a carton.snapshot file much like running pip freeze >> requirements.txt in Python.
• cpan-reporter is very easy to use to create CPAN Testers reports

Writing command line tools made easy (Tina Müller)

• good command line tools have:
  • usage output
  • usage error messages
  • man pages
• why write something new?
  • existing solutions don’t really fit together well
• App::Spec
  • define an app specification in a yaml file
  • will create a cli app for you, with bash completion, extended bash completion, help text, colours etc.
  • can define code which will be run when commands are run
  • there is also a version written in bash
• API::CLI
  • a tool which uses OpenAPI and App::Spec for making API calls.
  • e.g. can make calls to the DigitalOcean or GitHub APIs

Lightning talks

Goodbye screen, hello tmux (Tina Müller)

• different concepts
• screen
  • multiple windows
  • screen can be split into multiple windows
• tmux
  • several windows
  • windows split into multiple panes
• tutorial on GitHub https://tmux.github.io

fast as fsck (gonzo)

• showed some projects he’s worked on as part of the p5pclub at Booking
• many modules sped up using XS

Traits as plugins (dakkar)

• used roles to simplify an old program

sudo from a folder for the slightly trusted user (rgeoffrey)

• ways to get jenkins to run commands with sudo privs
• can specify a directory in /etc/sudoers
  • jenkins ALL:NOPASSWD /home/jenkins/sudoable/
  • now scripts in this directory can be run by jenkins with sudo privs

The values that open source taught me (Leon Timmermans)

• philosophies that open source taught him over time
• scientific computing does not have the same values as open source programming (e.g. “works for me”)
• “match the project that you work on match the values you have”

Veure update (Ovid)

• now called Tau Station
• Tau Station Blog
• alpha release will only include Sol and Alpha Centauri
• nonlinear missions
• awesome graphics, even though it’s a text based MMORPG

SEOPAN (Nigel Hamilton)

• search trails
• Google is an association engine rather than a search engine
  • a search for a problem leads to solution results
• CPAN is a solution or source of solutions
• how to get CPAN higher in Google’s rankings when users are searching for solutions to problems
• StackOverflow games Google to get StackOverflow’s results at the top of Google’s rankings
• the Perl community could try something similar in order to improve Perl’s visibility

Medium Perl (Dave Cross)

• mentions medium.com, nice design, huge audience, has tags
• http://medium.com/cultured-perl
  • online perl magazine
  • outside echo chamber
  • requested article submissions
  • will republish from other sources

Perl (nine)

• talk about community, focus on Perl, rather than just versions
• how should we promote Perl?
• food for thought