EzDevInfo.com

ruby net-ssh login shell

Is there any way i can get a login shell in ruby using net-ssh? Is that even possible?

By login shell i mean the ones the source /etc/profile..

Using Net SSH to execute commands with sudo

I've been trying to write a small library using Thor to help assist me in quick creating new projects and sites. I wrote this small method:

def ssh(cmd)
  Net::SSH.start( server_ip, user, :port => port) do |session|
    session.exec cmd
  end
end

to just assist me in running quick commands on remote servers when needed.

The problem is when I need to run a command under sudo on the remote end, the script just seems to hang on me. For example when executing this...

ssh("sudo cp #{file_from_path} #{file_to_path}" )

The script will prompt me for a password

[sudo] password for user:

But then the whole thing hhangs after typing it in.

Would anyone happen to know why it hangs exactly, and what I can do to run sudo command on a remote server under Net::SSH (or some alternative)?

*note: Before suggested, I had originally started writing this library as a recipe under Capistrano, until I came upon Thor, and thought it would be a good chance to try it out. I'm not against having to switch the whole thing back to Capistrano if needed, but I'd just be really surprised if there isn't an easy way to run sudo commands on a remote server.

Rails + Delayed jobs: SSH connections pool

Current situation: I have a rails app with N delayed job workers. Whenever I want to send an SSH request to some machine, I create a task for the worker. Worker performs something like:

Net:SSH.start(hostname, username, :password => pass) do |ssh|
  ssh.exec!(command)
end

Sometimes I create e.g. 50 such tasks to be performed one by one, or within 5-10 minutes from each other. Each task in this case opens separate connection, which is not effective and sometimes is blocked by target servers due to too many connections.

What I want: Have opened connections stored somewhere and reused by each worker. So that each worker obtains connection somehow and then just run

ssh.exec!(command)

What I have tried:

• storing connections in file/database/cache looks to be not possible as they are not serializable
• have tried using singleton class with global variables instantiated under initializers. However class object is different for each worker (later have found that global variables can not be an option).

Are there way to solve that? Any other ideas? Thanks!!

Localtunnel gem / Net::SSH not working

As per the instructions at http://progrium.com/localtunnel/ and elsewhere:

gem install localtunnel
localtunnel -k ~/.ssh/id_rsa.pub 3000

I get the error:

/path/to/gems/net-ssh-2.6.7/lib/net/ssh/transport/packet_stream.rb:103:in `block in next_packet': connection closed by remote host (Net::SSH::Disconnect)

It does not matter whether I include the key or not, or try different port numbers i.e.

localtunnel 3000

Any ideas?

Get Live logs in front end on rails application

Here in the rails 3.x app,I am using net::ssh and running some commands to remote pc.I would like to display the live logs to the user's browser.like,If two commands are running in net::ssh to execute i.e echo "Hello",echo "Bye" is passed then "Hello" should be displayed in the browser immediately finishing after its execution.Here is the code I am using for ssh connection and running commands in ruby on rails application

Net::SSH.start( @servers['local'] , @machine_name, :password => @machine_pwd, :timeout => 30) do |ssh|      
  ssh.open_channel do |channel|
    channel.request_pty         
    channel.exec("echo 'ssh started'")                                   
    channel.exec("ruby -v") 
    channel.exec("rails -v") 
    channel.exec("echo 'ssh Finished'") 


    channel.on_close do |ch|
      puts "****shell terminated****"
    end

    channel.on_eof do |ch|
      puts "****remote end is done sending data****"            
    end
    channel.on_extended_data do |ch, type, data|
      puts "****got stderr****: #{data.inspect}"
    end
    channel.on_data do |channel, data|
      puts "****ondata****: #{data} \n"                    
      puts "****channel****: #{channel} \n"          
      $logs << data # this data to be sent immediately to browser in each go                                  
    end

  ssh.loop      
  end
end  

Here on_data is sending data in every command execution,I need this data to be sent to the browser immediately.Is there any way to do this.So that I can achieve live logs in front end browser.Thanks!

How to net-ssh sudo su in Ruby

I'm trying to figure out how to a send chain of multiple net-ssh commands after a sudo su - #{su_user} in Ruby.

My current code is below, and hangs with the sudo su command, even after the send_data "#{password}\n".

Meanwhile, on the system, a manual execution of sudo su - admin2 does not require a password entry.

Any help would be appreciated!

require 'rubygems'  
require 'net/ssh'
host = 'hostA'

user = 'admin'
password = 'hostA_pwd'
su_user = 'Admin2'

Net::SSH.start(host, user, :password => password) do |ssh|

     ssh.open_channel do |channel|  
       channel.request_pty do |c, success|
         raise "could not request pty" unless success

         channel.exec "pwd; whoami; sudo su - #{su_user} ; pwd ; whoami"
         channel.on_data do |c_, data|
           if data =~ /\[sudo\]/ || data =~ /Password/i
             channel.send_data "#{password}\n"
           else       
             result << data
           end
         end
         puts result
       end
     end
     ssh.loop
  end

Error "non-absolute home" via Net:SSH

The code in question

Net::SSH.start('server name', 'user')

This returns "non-absolute home". The 'user' in fact does have a home directory. One suggested approach was to modify ~/.ssh/config with full paths to the IdentityFile. This did not solve the issue.

The crazy part of this is that the code works fine if called through irb or console. The moment we try calling it from within a class method (with the same code) it returns the "non-absolute home" error.

The 'user' can also ssh into the server via command line without issue. The server is running Ubuntu.

UPDATE

Thanks to @Phrogz - The fix for this was setting ENV['HOME'] to '/home/deploy'. However, I have not figured out why $HOME is getting set to "." on the server. So, I will leave this question up without an "Answer" until I, or someone else, figures that out. Having to manually set HOME feels more like a "hack" than a proper solution, but it does work.

Generate SSH Keypairs (private/public) without ssh-keygen

I'm working on a Ruby/Rack application that needs to generate SSH keypairs. As much as I'd like to call ssh-keygen from the application, I can't because it's designed to run on Heroku and they don't support calling that command.

I've been able to get private/public RSA keys using OpenSSL in the Ruby standard library doing the following:

key = OpenSSL::PKey::RSA.generate(2048)
# => -----BEGIN RSA PRIVATE KEY----- ....
key.public_key
# => -----BEGIN RSA PUBLIC KEY----- ....

Unfortunately an RSA public key and an SSH public key is not the same thing, even though they can be generated from the same RSA key. An SSH public key looks something like the following:

ssh-rsa AAAAB3NzaC1yc2EAAAABIwA.....

Is it possible to generate SSH keys or convert RSA keys to SSH in Ruby without using ssh-keygen?

How to get exit status with Ruby's Net::SSH library?

I have a snippet of code, simply trying to execute a script on a remote server, in the event that it fails, I'd like to make a follow-up call, imagine this:

require 'rubygems'
require 'net/ssh'
require 'etc'

server = 'localhost'

Net::SSH.start(server, Etc.getlogin) do |ssh|
  puts (ssh.exec("true")  ? 'Exit Success' : "Exit Failure")
  puts (ssh.exec("false") ? 'Exit Success' : "Exit Failure")  
end

I would expect (ignoring that stdout and stderr are printed in my contrived example) - but first line should exit with 0 which I would expect Ruby would interperate as false and display "Exit Failure" (sure, so the logic is wrong, the ternary needs to be flipped) - but the second line should exit with the opposite status, and it doesn't.

I can't even find anything in the documentation about how to do this, and I'm a little worried that I might be doing it wrong?!

Can't install chef, gem version conflict with net-ssh net-ssh-multi net-ssh-gateway

Using rvm, and an empty gemset, I get this:

$ gem install chef --no-ri --no-rdoc
ERROR:  While executing gem ... (Gem::DependencyError)
    Unable to resolve dependencies: chef requires net-ssh (~> 2.2.2); net-ssh-multi requires net-ssh (>= 2.6.5); net-ssh-gateway requires net-ssh (>= 2.6.5)

I've tried resolving it by installing earlier versions of net-ssh-gateway and net-ssh-multi, but net-ssh-multi version 1.1 confounds me by installing 1.1.2.

Cannot connect using keys with Ruby and net/ssh

I'm having trouble connecting via ssh from ruby using the 'net-ssh' gem, getting Net::SSH::AuthenticationFailed. The code is below

require 'net/ssh'
keys = ["path_to_private_key"]
Net::SSH.start('host', 'user',:keys => keys, :verbose => :debug) do |ssh|
  #ssh code

end

Using ssh directly from the command line works:

ssh -i <path_to_private_key> user@host

Do I have the the ssh API wrong? I have tried both 'user@host' and just 'user' as the username with the same result.

Here is the debugging output:

D, [2011-07-26T19:42:00.135148 #3511] DEBUG -- net.ssh.transport.session[140b040]: establishing connection to host:22
D, [2011-07-26T19:42:00.164190 #3511] DEBUG -- net.ssh.transport.session[140b040]: connection established
I, [2011-07-26T19:42:00.164344 #3511]  INFO -- net.ssh.transport.server_version[140a3fc]: negotiating protocol version
D, [2011-07-26T19:42:00.192092 #3511] DEBUG -- net.ssh.transport.server_version[140a3fc]: remote is `SSH-2.0-OpenSSH_5.3p1 Debian-3ubuntu3'
D, [2011-07-26T19:42:00.192177 #3511] DEBUG -- net.ssh.transport.server_version[140a3fc]: local is `SSH-2.0-Ruby/Net::SSH_2.1.4 x86_64-linux'
D, [2011-07-26T19:42:00.212348 #3511] DEBUG -- tcpsocket[140aaf0]: read 784 bytes
D, [2011-07-26T19:42:00.212461 #3511] DEBUG -- tcpsocket[140aaf0]: received packet nr 0 type 20 len 780
I, [2011-07-26T19:42:00.212515 #3511]  INFO -- net.ssh.transport.algorithms[13e9058]: got KEXINIT from server
I, [2011-07-26T19:42:00.212604 #3511]  INFO -- net.ssh.transport.algorithms[13e9058]: sending KEXINIT
D, [2011-07-26T19:42:00.212698 #3511] DEBUG -- tcpsocket[140aaf0]: queueing packet nr 0 type 20 len 556
D, [2011-07-26T19:42:00.212741 #3511] DEBUG -- tcpsocket[140aaf0]: sent 560 bytes
I, [2011-07-26T19:42:00.212771 #3511]  INFO -- net.ssh.transport.algorithms[13e9058]: negotiating algorithms
D, [2011-07-26T19:42:00.212859 #3511] DEBUG -- net.ssh.transport.algorithms[13e9058]: negotiated:
* kex: diffie-hellman-group-exchange-sha1
* host_key: ssh-rsa
* encryption_server: aes128-cbc
* encryption_client: aes128-cbc
* hmac_client: hmac-sha1
* hmac_server: hmac-sha1
* compression_client: none
* compression_server: none
* language_client: 
* language_server: 
D, [2011-07-26T19:42:00.212889 #3511] DEBUG -- net.ssh.transport.algorithms[13e9058]: exchanging keys
D, [2011-07-26T19:42:00.212998 #3511] DEBUG -- tcpsocket[140aaf0]: queueing packet nr 1 type 34 len 20
D, [2011-07-26T19:42:00.213033 #3511] DEBUG -- tcpsocket[140aaf0]: sent 24 bytes
D, [2011-07-26T19:42:00.292238 #3511] DEBUG -- tcpsocket[140aaf0]: read 152 bytes
D, [2011-07-26T19:42:00.292389 #3511] DEBUG -- tcpsocket[140aaf0]: received packet nr 1 type 31 len 148
D, [2011-07-26T19:42:00.297526 #3511] DEBUG -- tcpsocket[140aaf0]: queueing packet nr 2 type 32 len 140
D, [2011-07-26T19:42:00.297627 #3511] DEBUG -- tcpsocket[140aaf0]: sent 144 bytes
D, [2011-07-26T19:42:00.321754 #3511] DEBUG -- tcpsocket[140aaf0]: read 720 bytes
D, [2011-07-26T19:42:00.321909 #3511] DEBUG -- tcpsocket[140aaf0]: received packet nr 2 type 33 len 700
D, [2011-07-26T19:42:00.326688 #3511] DEBUG -- tcpsocket[140aaf0]: queueing packet nr 3 type 21 len 20
D, [2011-07-26T19:42:00.326793 #3511] DEBUG -- tcpsocket[140aaf0]: sent 24 bytes
D, [2011-07-26T19:42:00.326927 #3511] DEBUG -- tcpsocket[140aaf0]: received packet nr 3 type 21 len 12
D, [2011-07-26T19:42:00.327325 #3511] DEBUG -- net.ssh.authentication.session[1475c10]: beginning authentication of `user@host'
D, [2011-07-26T19:42:00.327519 #3511] DEBUG -- tcpsocket[140aaf0]: queueing packet nr 4 type 5 len 28
D, [2011-07-26T19:42:00.327604 #3511] DEBUG -- tcpsocket[140aaf0]: sent 52 bytes
D, [2011-07-26T19:42:00.461124 #3511] DEBUG -- tcpsocket[140aaf0]: read 52 bytes
D, [2011-07-26T19:42:00.461199 #3511] DEBUG -- tcpsocket[140aaf0]: received packet nr 4 type 6 len 28
D, [2011-07-26T19:42:00.461260 #3511] DEBUG -- net.ssh.authentication.session[1475c10]: trying publickey
D, [2011-07-26T19:42:00.461370 #3511] DEBUG -- net.ssh.authentication.agent[1470800]: connecting to ssh-agent
E, [2011-07-26T19:42:00.461423 #3511] ERROR -- net.ssh.authentication.agent[1470800]: could not connect to ssh-agent
D, [2011-07-26T19:42:00.461521 #3511] DEBUG -- net.ssh.authentication.methods.publickey[1470bfc]: trying publickey (d7:62:48:07:23:1d:c0:de:80:0d:27:a3:ab:92:72:be)
D, [2011-07-26T19:42:00.461610 #3511] DEBUG -- tcpsocket[140aaf0]: queueing packet nr 5 type 50 len 364
D, [2011-07-26T19:42:00.461648 #3511] DEBUG -- tcpsocket[140aaf0]: sent 388 bytes
D, [2011-07-26T19:42:00.486881 #3511] DEBUG -- tcpsocket[140aaf0]: read 68 bytes
D, [2011-07-26T19:42:00.487057 #3511] DEBUG -- tcpsocket[140aaf0]: received packet nr 5 type 51 len 44
D, [2011-07-26T19:42:00.487186 #3511] DEBUG -- net.ssh.authentication.session[1475c10]: allowed methods: publickey,password
D, [2011-07-26T19:42:00.487261 #3511] DEBUG -- net.ssh.authentication.session[1475c10]: trying password
E, [2011-07-26T19:42:00.487346 #3511] ERROR -- net.ssh.authentication.session[1475c10]: all authorization methods failed (tried publickey, password)

Thanks

Table files transfered between servers flags table as crashed

Work has a web site that uses large data sets, load balanced between two MySQL 5.6.16-64.2 servers using MyISAM, running on Linux (2.6.32-358.el6.x86_64 GNU/Linux.) This data is being updated hourly from a text based file set that is received from a MS-SQL database. To avoid disruption on reads from the web site and at the same time make sure the updates doesn't take too long following process was put in place:

Have the data one a third Linux box (only used for data update processing,) update the different data tables as needed, move a copy of the physical table files to the production servers under a temporary name, and then do a table swap by MySQL TABLE RENAME.

But every time the table (under the temporary name) is seen by the destination MySQL servers as being crashed and require repair. The repair takes too long, so it cannot be forced to do a repair before doing the table swap.

The processing is programmed in Ruby 1.8.7 by having a thread for each server (just as a FYI, this also happens if not doing it in a thread to a single server.)

The steps to perform file copy is as follows:

Use Net::SFTP to transfer the files to a destination folder that is not the database folder (done due to permissions.) Code example of the file transfer for the main table files (if table also has partition files then they are transferred separately and rspFile is assigned differently to match the temporary name.) For speed it is parallel uploaded:

Net::SFTP.start(host_server, host_user, :password => host_pwd) do |sftp|
  uploads = fileList.map { |f|
    rcpFile = File.basename(f, File.extname(f)) + prcExt + File.extname(f)
    sftp.upload(f, "/home/#{host_user}/#{rcpFile}")
  }
  uploads.each { |u| u.wait }
end

Then assign the files the owner and group to the mysql user and to move the files to the MySQL database folder, by using Net::SSH to execute sudo shell commands:

Net::SSH.start(host_server, host_user, :port => host_port.to_i, :password => host_pwd) do |ssh|
  doSSHCommand(ssh, "sudo sh -c 'chown mysql /home/#{host_user}/#{prcLocalFiles}'", host_pwd)
  doSSHCommand(ssh, "sudo sh -c 'chgrp mysql /home/#{host_user}/#{prcLocalFiles}'", host_pwd)
  doSSHCommand(ssh, "sudo sh -c 'mv /home/#{host_user}/#{prcLocalFiles} #{host_path}'", host_pwd)
end

The doSSHCommand method:

def doSSHCommand(ssh, cmd, pwd)
  result = ""
  ssh.open_channel do |channel|
    channel.request_pty do |c, success|
        raise "could not request pty" unless success

      channel.exec "#{cmd}" do |c, success|
          raise "could not execute command '#{cmd}'" unless success

        channel.on_data do |c, data|
          if (data[/\[sudo\]|Password/i]) then
            channel.send_data "#{pwd}\n"
          else
            result += data unless data.nil?
          end
        end
      end
    end
  end
  ssh.loop

  result
end

If done manually by using scp to move the files over, do the owner/group changes, and move the files, then it never crashes the table. By checking the file sizes compared between scp and Net::SFTP there are no difference.

Other process methods has been tried, but experience they take too long compared to using the method described above. Anyone have an idea of why the tables are being crashed and if there a solution to avoid table crash without having to do a table repair?

Run MySQL query over multiple SSH tunnels?

I have a situation somewhat similar to "How to create a ssh tunnel in ruby and then connect to mysql server on the remote host."

From my local machine, I want to run a query against a production MySQL database host. My local machine cannot directly connect to the database host. However, if I SSH to a gateway machine, I can run the MySQL client and connect to the database host.

I'm trying to figure out how to programmatically to run a query from my machine that is forwarded to the gateway machine, which forwards it to the database server, and have results returned. Basically I want to have LOCAL => GATEWAY => DB_HOST forwarding.

I have a hacky solution that runs ssh.exec and MySQL on the command line, shown below, but, I'm trying to find a way that does port forwarding twice. I tried, but haven't been successful so far.

require 'rubygems'  
require 'mysql2'  
require 'net/ssh/gateway' 

gateway = Net::SSH::Gateway.new(
  $gateway_host, $gateway_user,:password => $gateway_pass)

# This works
ssh = gateway.ssh($db_host, $db_login_user)
data = ssh.exec!("mysql -u#{$db_user} -p#{$db_pass} #{$db_name} -e '#{$sql_query}'")
puts "#{data.class} is the class type"
puts data

# Want to do something like this with port forwarding
# client = Mysql2::Client.new(:host => $db_host,
#                            :username => $db_user,
#                            :password => $db_pass,
#                            :database => $db_name,
#                            :port => port)
# results = client.query($sql_query)

puts "end stuff"
ssh.close

Any suggestions?

Browser based (free to use) SSH

I have a commercial product that allows users to connect to various SSH end-points. Currently these users are forced to download and use Putty... Seems pretty straightforward, except that my SSH end-points require RSA/Private Key authentication. So now connectivity to these end-points is becoming a pain, because I need to explain to my users how to: 1) Download and configure Putty. 2) Manage, configure and use their PEM private keys. I would like to make everything transparent by 'just working' through the browser. I own all information (both IP addresses and PEM connectivity keys), so is there such a thing as a browser based SSH that is both capable and can access RSA keys for connectivity?

Ruby NET::SCP containing Wildcard

I need to download a file daily from a client that I have SCP but not SSH access to.

The file name will always be /outgoing/Extract/visit_[date]-[timestamp].dat.gz'

For example yesterdays file was called visits_20130604-090003.dat.gz

I can not rely on the fact that the time stamp will always be the same, but the date should always be yesterdays date:

My set up so far:

My home directory contains to sub-directories named downloads_fullname and downloads_wildcard.

It also contains an simple ruby script named foo.rb.

The contents of foo.rb are this`

#! /usr/bin/ruby
require 'net/ssh'
require 'net/scp'
yesterday = (Time.now - 86400).strftime('%Y%m%d')

Net::SCP.start('hostname', 'username') do |scp|
  scp.download!('/outgoing/Extract/visits_' + yesterday + '-090003.dat.gz', 'downloads_fullname')
  scp.download!('/outgoing/Extract/visits_' + yesterday + '-*.dat.gz', 'downloads_wildcard')
end

When run the downloads_fullname directory contains the file, but the downloads_wildcard directory does not.

Is there any way to use wildcarding in Net::SCP? Or does anybody have any sly workarounds? I tried \*to no avail.