Stories
Slash Boxes
Comments

SoylentNews is people

posted by martyb on Tuesday January 28 2020, @05:00AM   Printer-friendly
from the See-python.-See-python-go.-Go-python!-Go!-Go!-Go! dept.

Go compared to Python for small scale system administration scripts and tools:

We write a certain amount of scripts and tools around here. I like Go and have used it for a while, we have some tools already written in Go, and while I'm also a long term user of Python I'm on record as being unhappy with various developments around Python 3. Despite all of this, Python is the programming language I default to when I need to do something that's more complicated than a shell script (and not because of our policy on internal tools). Over time I've come to believe that Python has some important pragmatic properties in our sort of relatively small scale environment with generally modest use of local tools, despite Go's collection of appealing properties.

The first useful property Python has is that you can't misplace the source code for your deployed Python programs. Unless you do something very peculiar, what you deploy is the source code (well, a version of it). With Go you deploy a compiled artifact, which means that you may someday have to find the source code and then try to match your compiled binary up against some version of it. Of course your deployed Python program can drift out of sync with the master copy in your version control repository, but sorting that out only requires use of diff.

Closely related to this is that Python code is generally simple for people to modify and re-deploy. Modest Python scripts and tools are likely to be only a single .py file, which you can edit and copy around, and even somewhat bigger ones are likely to just be a directory that can be copied. Deploying Go code requires not just the correct source code but also a Go development environment and the knowledge of how to build Go programs from source. With Python, you can even try things out by just modifying the deployed version in place, then back-port your eventual changes to the official master copy in your version control system.

(These days Go's support for modules makes all of this simpler than it used to be, but there are still important considerations and some potential complexities.)


Original Submission

This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
(1)
  • (Score: 5, Insightful) by The Mighty Buzzard on Tuesday January 28 2020, @06:22AM (4 children)

    by The Mighty Buzzard (18) Subscriber Badge <themightybuzzard@proton.me> on Tuesday January 28 2020, @06:22AM (#949938) Homepage Journal

    No shit, Sherlock. Any *NIX admin could have told you that you don't write chincy little local tools in a compiled language. If it's simple enough for a shell script, you use a shell script. If you need a little more horsepower language-wise, you use a scripting language. You never use a compiled language on a local tool that you'll personally be using and debugging unless performance becomes an issue.

    --
    My rights don't end where your fear begins.
    • (Score: 0, Insightful) by Anonymous Coward on Tuesday January 28 2020, @11:02AM (1 child)

      by Anonymous Coward on Tuesday January 28 2020, @11:02AM (#950007)

      don't write chincy little local tools in a compiled language.

      https://stackoverflow.com/questions/41542941/is-it-possible-to-run-go-code-as-a-script [stackoverflow.com]

      or do you mean something like python? Because python compiles things too...

      https://stackoverflow.com/questions/5607283/how-can-i-manually-generate-a-pyc-file-from-a-py-file [stackoverflow.com]

      Also, utilities like `ls` are little tools written in C, which is a very compiled language. So not sure.....

      • (Score: 2) by The Mighty Buzzard on Wednesday January 29 2020, @01:13AM

        by The Mighty Buzzard (18) Subscriber Badge <themightybuzzard@proton.me> on Wednesday January 29 2020, @01:13AM (#950370) Homepage Journal

        Crucial tools that come as part of the operating system or that are used in many organizations around the world on hardware with incredible variety of resources have a valid case for being compiled. Minor tools that will never see the outside of your company, or in many cases even a single server on your company, do not.

        --
        My rights don't end where your fear begins.
    • (Score: 0) by Anonymous Coward on Tuesday January 28 2020, @05:09PM (1 child)

      by Anonymous Coward on Tuesday January 28 2020, @05:09PM (#950132)

      Seconded.

      In general, unless performance is a concern I favor scripting languages for that reason. "What is this piece of code doing?" "Well gee, I guess I'll open the file right here and look!"

      I'm actually a fan of Raku language (formerly known as Perl 6), but it's not popular enough to use in a work environment. You have to stick with the tools your peers know on any important project.

  • (Score: 2, Insightful) by Anonymous Coward on Tuesday January 28 2020, @06:27AM

    by Anonymous Coward on Tuesday January 28 2020, @06:27AM (#949940)

    The choice of tools is of course personal and depending on the work but I would use neither Python nor Go for small scale system administration scripts and tools. I would just use shell scripts. Shell scripts are powerful enough for the administration tasks I do and their simplicity is winning for me. If I would pick other programming language I would end up importing a bunch of libraries (and in worst case installing modules by something like pip install) to do things I could do directly with a simple shell script.

    For me it is about right tool for right task. In the same way I would never use shell scripts to build a bigger application, I would not use a "normal" programming language for something where shell scripts fit better.

  • (Score: 4, Interesting) by edIII on Tuesday January 28 2020, @06:28AM (1 child)

    by edIII (791) on Tuesday January 28 2020, @06:28AM (#949941)

    Not a fan of the language selections. Python isn't a great language, and there is plenty of discussions regarding the drawbacks from seasoned programmers. Same for PHP, but I digress.

    The number one consideration these days is security. If it's not, your bent over, buttered up, and just waiting for it. The biggest challenge with writing scripts like these is not opening up gaping security holes. Go is a compiled language, and I believe has a better reputation for security, in both code and the repositories. I would have to lean towards Go to write tools.

    Search for major hacks involving "submarine" like attacks where a vulnerability entered a major corporations tool chain through the Python repository. Which is why I come back to Perl. On OpenBSD at least, the Perl packages are vetted each time there is a new release (IIRC). Which is above and beyond the repository, CPAN. I've not heard of any attacks on it where malicious code infected a module. It's all very seasoned code too. The chance of a vuln entering CPAN, and then getting past an OpenBSD contributor is slim. As far as the environment goes, there is a slight learning curve I'll admit. Once you got it down, it's extremely easy. I have a BASH script that bootstraps the Perl environment and installs the core modules I need for my tools. Perl is easier to write with than Go, being a scripted language IMO. Which might make it easier to understand by future developers.

    In the end I want to write system tools using something the least likely to uptake malicious code from untrusted sources.

    --
    Technically, lunchtime is at any moment. It's just a wave function.
    • (Score: 1, Interesting) by Anonymous Coward on Tuesday January 28 2020, @03:24PM

      by Anonymous Coward on Tuesday January 28 2020, @03:24PM (#950087)

      Exactly.
      We got caught by the Python 2 forced obsolescence and decided that, as we'd have to rewrite the whole thing(*), we should choose a language where the stable branch is still maintained.

      Perl was chosen without enthusiasm, since all alternatives had some big flaw or other.

      Surprisingly, as in often the case with C++, a judicious choice in the subset of the language that one use and a common coding style have both given us a very stable, readable and maintainable option.

      (*) A lot of our tools are processing data and are perfectly aware of the various encodings. Python 3's "helpfull" concept of Unicode was a real hell, and 2to3 failed on _every_ source module.

  • (Score: 1, Informative) by Anonymous Coward on Tuesday January 28 2020, @11:04AM

    by Anonymous Coward on Tuesday January 28 2020, @11:04AM (#950008)

    Use what is popular. Use what is used already out there. Don't write stuff in language no one is using.

    Go is a little popular on container end, but I wouldn't exactly want to be programming in it. I would rather write stuff in Javascript (like Node) or Python or even Perl. Niche languages like Erlang or Rust ... avoid. At least if you want not to be hated by someone that will have to maintain your code

  • (Score: 2, Interesting) by PiMuNu on Tuesday January 28 2020, @11:08AM

    by PiMuNu (3823) on Tuesday January 28 2020, @11:08AM (#950009)

    Who owns the language? I assume that it is free as in beer; are there any copyright issues? Remember, API may be copyrightable now. Is governance owned by Google, and what are the implications?

  • (Score: 3, Interesting) by Subsentient on Tuesday January 28 2020, @11:49AM (1 child)

    by Subsentient (1111) on Tuesday January 28 2020, @11:49AM (#950020) Homepage Journal

    Go is such an odd language, they're trying to use it as a compiled language without pointer math or unsafe code blocks, and Go can't function in kernel space. I guess you could do that, but Go has much poorer bindings than something like Python will.

    Python 3, especially lately, is pretty good. I have several Python scripts for my sysadmin stuff. I recommend it over Go any day.
    Python is probably the only "popular"/trendy language out there right now that actually deserves it, come to think of it.

    --
    "It is no measure of health to be well adjusted to a profoundly sick society." -Jiddu Krishnamurti
    • (Score: 2) by HiThere on Tuesday January 28 2020, @04:42PM

      by HiThere (866) Subscriber Badge on Tuesday January 28 2020, @04:42PM (#950123) Journal

      Go code *should* execute faster, and it's a lot easier to write concurrent go programs. But it doesn't do GUIs.

      Python comes with "tools included", and lots more optional libraries.

      I wanted to use go, but various considerations meant I ended up choosing Python. (FWIW, I prefer Python3 over Python2, but the preference is marginal so far. Once the type annotations become part of the language, it will be a lot less marginal.)

      --
      Javascript is what you use to allow unknown third parties to run software you have no idea about on your computer.
  • (Score: 0, Offtopic) by Anonymous Coward on Tuesday January 28 2020, @12:01PM (2 children)

    by Anonymous Coward on Tuesday January 28 2020, @12:01PM (#950026)

    Use Go and your code will be sold to law enforcement agencies.

  • (Score: 1, Insightful) by Anonymous Coward on Tuesday January 28 2020, @02:04PM

    by Anonymous Coward on Tuesday January 28 2020, @02:04PM (#950054)

    Same mentality from the programmers. Same language issues.

  • (Score: 1, Interesting) by Anonymous Coward on Tuesday January 28 2020, @02:06PM (1 child)

    by Anonymous Coward on Tuesday January 28 2020, @02:06PM (#950056)

    In my field (device testing), I have used Python (also a homebrew scripting language) and C# for testing.

    I strongly feel that testing should be done with a scripting language as:
    1) run speed is typically limited by the interface, not the language
    2) the lack of a binary means that it is simple to determine what version of your source was used to test the device.

    I definitely preferred Python as a test language even in a deployment that was larger than many people think you should do with Python (hundreds of thousands of lines of code, most of it test settings).

    I know many do not like the dynamic typing of Python, and I will admit it is a double edged sword (how many times I got caught parsing a file char by char instead of line by line), but being able to put things in containers that weren't specifically designed to hold those things is, at times, very useful.

    • (Score: 0) by Anonymous Coward on Wednesday January 29 2020, @03:53AM

      by Anonymous Coward on Wednesday January 29 2020, @03:53AM (#950448)

      Test code also tends to be very simple in structure.
      Not saying it is easy to come up with good tests -- it's not -- but the code itself isn't doing anything complicated.
      For use cases like this, you don't need a powerful language, and the overhead of setting things up (type declarations, etc.) before using them is painful and doesn't buy you anything. Variables are few and short lived; data structures are not custom defined, but rather just hashtables, strings, and maybe integers. What does typesafety mean if you only use the same 3 types in ad hoc code? It is unnecessary.

  • (Score: 0) by Anonymous Coward on Tuesday January 28 2020, @07:24PM (6 children)

    by Anonymous Coward on Tuesday January 28 2020, @07:24PM (#950191)

    One thing that is a constant headache with Python is packaging and deployment.
    Its roots as a simple scripting language really show here, unfortunately.
    Importing other python modules and deploying them is loaded with PATH and other env var issues and other low level OS-dependent flakiness.
    At work we were using Python virtual environments, but what a brittle hack when you have multiple Python executables up and running at the same time. We Dockerized everything and frankly I see that as the only viable solution to the Python deployment problem. Single scripts that run in a linear fashion and terminate in short order seem to be the only relatively troublefree scenario for Python deployment otherwise.

    • (Score: 2) by The Mighty Buzzard on Wednesday January 29 2020, @01:18AM (5 children)

      by The Mighty Buzzard (18) Subscriber Badge <themightybuzzard@proton.me> on Wednesday January 29 2020, @01:18AM (#950374) Homepage Journal

      You must hate the shit out of shared libraries as well then.

      --
      My rights don't end where your fear begins.
      • (Score: 0) by Anonymous Coward on Wednesday January 29 2020, @03:40AM (4 children)

        by Anonymous Coward on Wednesday January 29 2020, @03:40AM (#950443)

        Have you ever had to deploy a system comprised of multiple Python servers, each of which has its own complicated dependencies of other Python source files, shared libraries, Python wheels, RPM packages, etc.?

        I am guessing that's a big NO.

        Python apps (not chintzy standalone scripts) are a giant hairball of every fucking type of code you can slap onto your machine. It's complete chaos, and the only way to avoid clashes and make for reproducible builds is to isolate each Python server and all its necessary supporting shit into its own little container that can't clash with other programs/libraries on the machine. There is no other way to realistically achieve this.

        • (Score: 2) by The Mighty Buzzard on Wednesday January 29 2020, @05:09AM (3 children)

          by The Mighty Buzzard (18) Subscriber Badge <themightybuzzard@proton.me> on Wednesday January 29 2020, @05:09AM (#950501) Homepage Journal

          We currently have Gentoo, Centos, and multiple versions of Ubuntu servers for SN. You tell me if I've had headaches.

          And I would have had even worse headaches with shared libs if we tried to compile shit for each of them.

          --
          My rights don't end where your fear begins.
          • (Score: 0) by Anonymous Coward on Wednesday January 29 2020, @01:11PM (2 children)

            by Anonymous Coward on Wednesday January 29 2020, @01:11PM (#950606)

            Heh heh. If you make a Docker container for a compiled app, the container gets its own copies of any shared libs it uses and nothing outside the container can see those libs, so no clashes. You *effectively* turn a shared lib into a statically compiled lib. ^_^

            • (Score: 2) by The Mighty Buzzard on Wednesday January 29 2020, @01:22PM

              by The Mighty Buzzard (18) Subscriber Badge <themightybuzzard@proton.me> on Wednesday January 29 2020, @01:22PM (#950609) Homepage Journal

              True, and you get massive overhead in exchange. This is not about full applications that might warrant such though but things on the scope of system administration scripts that most certainly do not.

              --
              My rights don't end where your fear begins.
            • (Score: 0) by Anonymous Coward on Wednesday January 29 2020, @08:59PM

              by Anonymous Coward on Wednesday January 29 2020, @08:59PM (#950843)

              Here is how I think it went:

              1: Hey so, we keep getting incompatibilities of the various libraries we run on.

              2: Why don't you statically compile it then, that way all the libraries are built in.

              1: No, statically compiled binaries are too big. If only there were some way to link the executable to the libraries it uses.

              2: ...

              1: Yeah, we can ship the libraries with the binary. But some might be missing.

              2: ...

              1: Oh! we can just include the whole OS base image in there. But what about updates.

              2: ...

              1: Oh! we can package them together in a unvetted public repo and never clean up old versions on updates.

              2: Isn't that just a fancy statically linked binary but with way more headache?

              1: Pft, It's like you don't know what you are talking about. The binary uses shared libs and will be much smaller. Now wait a minute while I start finding investors.

  • (Score: 2) by darkfeline on Wednesday January 29 2020, @04:19AM (2 children)

    by darkfeline (1030) on Wednesday January 29 2020, @04:19AM (#950470) Homepage

    > Closely related to this is that Python code is generally simple for people to modify and re-deploy. Modest Python scripts and tools are likely to be only a single .py file, which you can edit and copy around, and even somewhat bigger ones are likely to just be a directory that can be copied.

    This is fine for a tiny fleet of servers, but that sounds like a great way to end up with hundreds of subtly different versions of every script on every server (the accursed snowflake server). Good luck figuring out why these specific five lines in this specific script was changed on these specific 14 servers, when the original sysadmin who made that change may no longer be at the company.

    The fact that you need a sane deploy process for Go is an advantage in my opinion. The author of the article sounds like a self-taught electrician who thinks civil regulations are a bother because "it just works for me" until the day he burns down his house. All you need for a "Go development environment" is downloading the latest release, unpacking it, and running the included statically linked go command (assuming your distribution doesn't already package it). If that's too big of a hurdle for the author, it sounds like they have much more systemic issues to deal with than "Go is kind of hard to use".

    Every design decision in Go was made for a very pragmatic reason, including the practical reality that you have to deal with other people. (and as we all know, other people is hell)

    --
    Join the SDF Public Access UNIX System today!
    • (Score: 1, Insightful) by Anonymous Coward on Wednesday January 29 2020, @04:41AM

      by Anonymous Coward on Wednesday January 29 2020, @04:41AM (#950489)

      If you are in the position to copy the same statically linked binary to all your servers means that you are in the position to copy the same python script to all your servers.

    • (Score: 2) by The Mighty Buzzard on Wednesday January 29 2020, @05:16AM

      by The Mighty Buzzard (18) Subscriber Badge <themightybuzzard@proton.me> on Wednesday January 29 2020, @05:16AM (#950504) Homepage Journal

      AC's right. Nothing about compiled code makes it easier to distribute to the exact same machines. And if those machines are running disparate operating systems and versions thereof, you will have shared library version issues that are much harder to solve than scripting language library issues.

      --
      My rights don't end where your fear begins.
  • (Score: 0) by Anonymous Coward on Wednesday January 29 2020, @07:35AM (1 child)

    by Anonymous Coward on Wednesday January 29 2020, @07:35AM (#950547)

    Having several options - shell scripting, Python and Go - to choose from is good. If you don't want people fiddling in the code then compile. But note the comments on here about who owns Go. It depends on the situation and usage. Finding the right balance to the "mixed media" solution is the art.

(1)