Welcome to zewaren.net. This site presents myself and mostly archives the solutions to some problems I once had.

How to install OpenERP 8 (Odoo) on FreeBSD and run it alongside OpenERP 7.x

ImageImageThe FreeBSD logo

Situation

You're running a nice OpenERP 7.x on one of your favorite FreeBSD server. You'd like to try the new version 8 (Odoo), but you heard about the fact that databases can't be migrated automatically, so you want to run the two versions alongside.

Install version 8

Download and install

These ports should already be installed since you were running openerp-7.x previously. If not, install or upgrade them:

/graphics/py-imaging
/textproc/py-libxml2
/devel/py-pytz
/lang/py-mx-base
/databases/py-psycopg2
/graphics/py-chart
/graphics/py-pydot
/textproc/py-libxslt
/devel/py-lxml
/textproc/py-xml
/devel/py-yaml
/textproc/py-mako
/devel/py-dateutil
/print/py-reportlab2

Fetch and extract the release:

fetch 'http://nightly.odoo.com/8.0/nightly/src/odoo_8.0rc1-latest.tar.gz'
tar xzvf odoo_8.0rc1-latest.tar.gz
cd openerp-8.0rc1

Install the release

python setup.py install

Warning: this setup will install python packages itself without using the port tree. This is not optimal but it should work quite well anyway.

I recommend installing Odoo in a jail so that you don't trash your entire superb production system.

Set the permissions on the install folder:

chown -R openerpd:wheel /usr/local/lib/python2.7/site-packages/openerp-8.0rc1-py2.7.egg/

Create the configuration files

Theses configuration files might include some useless fields or some things that aren't supported anymore. Use them at your own risks.

/usr/local/etc/openerp-server.conf:

[options]
# Basic daemon configuration options ##################
root_path = /usr/local/lib/python2.7/site-packages/openerp-8.0rc1-py2.7.egg
addons_path = /usr/local/lib/python2.7/site-packages/openerp-8.0rc1-py2.7.egg/openerp/addons
# netinterface =
# interface =
xmlrpc_interface =
xmlrpc_port = 8169
debug_mode = False
stop_after_init = False
soap = False
xmlrpc = True
netrpc = True
netrpc_interface =
netrpc_port = 8070
secure = False
cache_timeout = 100000
pidfile = /var/run/openerp/openerp-server.pid
reportgz = False
admin_passwd = admin
login_message = False
price_accuracy = 2
csv_internal_sep = ,
translate_modules = ['all']

# Logging options #####################################
syslog = False
log_level = info
logfile = /var/log/openerp/openerp-server.log
assert_exit_level = warn

# Database options ####################################
db_name = openerp
db_user = openerp
db_password = openerppassword
db_host = localhost
db_maxconn = 64
pg_path = /usr/local/bin
list_db = True
# import_partial =

# SMTP options ########################################
smtp_server = localhost
smtp_port = 25
smtp_user = False
smtp_password = False
email_from = False

# Use demo files? #####################################
# without_demo = True
# demo = {}

/usr/local/etc/openerp-web.conf:

# Example configuration file for OpenERP-web
#
# This is an example configuration file, just copy it
# to openerp-web.conf and edit it to suit your needs.

# OpenERP Web server
[global]

# the ip and port the web server will be listening on
server.socket_host = "0.0.0.0" # 0.0.0.0 means all ip addresses
server.socket_port = 8180

# Sets the number of threads the server uses
server.thread_pool = 10

server.environment = "development"

# Simple code profiling
server.profile_on = False
server.profile_dir = "profile"

# Set to True if you are deploying your App behind a proxy
# e.g. Apache using mod_proxy
#tools.proxy.on = True

# If your proxy does not add the X-Forwarded-Host header, set
# the following to the *public* host url.
#tools.proxy.base = 'http://mydomain.com'

# logging
log.access_file = "/var/log/openerp-web/access.log"
log.error_file = "/var/log/openerp-web/error.log"

# OpenERP Server information
[openerp]
host = 'localhost'
port = '8170'
protocol = 'socket'

# Web client settings
[openerp-web]
# filter dblists based on url pattern?
# NONE: No Filter
# EXACT: Exact Hostname
# UNDERSCORE: Hostname_
# BOTH: Exact Hostname or Hostname_

dblist.filter = 'NONE'

# whether to show Databases button on Login screen or not
dbbutton.visible = True

# will be applied on company logo
company.url = ''

# options to limit data rows in M2M/O2M lists, will be overriden
# with limit="5", min_rows="5" attributes in the tree view definitions
child.listgrid.limit = 5
child.listgrid.min_rows = 5

Don't forget to do what's needed to allow user openerpd to write its log files.

Create the RC scripts

Don't forget to chmod them so they can be executed.

/usr/local/etc/rc.d/openerp-server:

#!/bin/sh
#
# PROVIDE: openerpd
# REQUIRE: postgresql LOGIN
#
# Add the following lines to /etc/rc.conf to enable openerp-server
#
#
# openerpd_enable (bool): Set to "NO" by default,
#                         Set it to "YES" to enable openerp-server
#
# openerpd_config (str):  The path to the openerp-server configuration file
#                         (defaults to /usr/local/etc/openerp-server.conf)
#
# openerpd_flags (str):   Extra arguments to be used when invoking
#                         the openerp-server daemon.
#
# Patch submitted by (c) Franck Porcher, Ph.D, to fix some issues regarding
# pidfile, log files, and discrepencies with the main configuration file.

#  getval varname file [default_value] [separator_char]
#  - Discard comment lines (any text leading with blanks then #)
#  - Retain only the last value set


. /etc/rc.subr

name=openerpd
command=/usr/local/bin/openerp-server
rcvar=openerpd_enable

load_rc_config $name

openerpd_enable="${openerpd_enable-"NO"}"
openerpd_config="${openerpd_config-"/usr/local/etc/openerp-server.conf"}"
openerpd_user="${openerpd_user-"openerpd"}"
openerpd_pidfile="${openerpd_pidfile:-"$(grep pidfile /usr/local/etc/openerp-server.conf | awk -F "=" ' { print $2 } ' |sed 's/[ ]//g' )"}"
openerpd_logdir="${openerpd_logdir:-"$(dirname `grep logfile /usr/local/etc/openerp-server.conf | awk -F "=" ' { print $2 } ' `)"}"
openerpd_flags="${openerpd_flags:-"--config=${openerpd_config}"}"

# /etc/rc.subr use $pidfile (not ${name}_pidfile)
pidfile="${openerpd_pidfile}"

required_files="${openerpd_config}"

start_precmd="${name}_prestart"
stop_cmd="${name}_stop"
status_cmd="${name}_status"
getval_cmd="${name}_getval"

openerpd_prestart()
{
    local d

    d="$(dirname "${openerpd_pidfile}")"
    if [ ! -d "${d}" ]
    then
        mkdir -p "${d}"
    fi
    chown "${openerpd_user}" "${d}"

    d="$openerpd_logdir"
    if [ ! -d "${d}" ]
    then
        mkdir -p "${d}"
    fi
    chown "${openerpd_user}" "${d}"
}

openerpd_stop()
{
    # echo "Looking for (openerp-server ${openerpd_flags})"
    openerpd_pid=$(pgrep -f "openerp-server ${openerpd_flags}")
    # Try its best to stop the service
    if [ -f "${openerpd_pidfile}" ] && [ "${openerpd_pidfile}" = "${openerpd_pid}" ]
    then
        echo "Stopping ${name}."
        kill -15 "$(cat "${openerpd_pidfile}")"
    else
        if [ -n "${openerpd_pid}" ]
        then
            echo "Stopping ${name}."
            kill -15 "${openerpd_pid}"
        else
            echo "${name} not running? (pidfile not found)"
        fi
    fi
}

openerpd_status()
{
    # Try its best to find the service's status
    if [ -f "${openerpd_pidfile}" ]
    then
        openerpd_pid="$(cat "${openerpd_pidfile}")"
    fi

    if [ -z "${openerpd_pid}" ]
    then
    openerpd_pid=$(pgrep -f "openerp-server.py ${openerpd_flags}")
    [ -n "${openerpd_pid}" ] && echo "${openerpd_pid}" > "${openerpd_pidfile}"
    fi

    if [ -n "${openerpd_pid}" ]
    then
        echo "${name} running with pid: $openerpd_pid"
    else
        echo "${name} not running? (pid not found)"
    fi
}

command_args=" >/dev/null 2>&1 &"

load_rc_config $name
run_rc_command "$1"

/etc/rc.conf:

openerpd_enable="YES"
openerpweb_enable="YES"
openerpweb_user="openerpd"

/usr/local/bin/openerp-server:

#!/usr/local/bin/python
# EASY-INSTALL-SCRIPT: 'openerp==8.0rc1','openerp-server'
import os
os.environ['PYTHON_EGG_CACHE'] = '/home/openerpd/.python-eggs'
os.environ['XDG_DATA_HOME'] = '/home/openerpd/.local'
__requires__ = 'openerp==8.0rc1'
import pkg_resources
pkg_resources.run_script('openerp==8.0rc1', 'openerp-server')

Start the server

/usr/local/etc/rc.d/openerp-server start

If the server does not start, try to start it manually and look for any output:

su -m openerpd -c 'sh -c "/usr/local/bin/openerp-server --config=/usr/local/etc/openerp-server.conf"'

Run OpenERP 7.x at the same time

Install OpenERP 7.x the same way as version 8, or using port finance/openerp-server.

Create the configuration files

Makes sure that the two version listen on different ports.

/usr/local/etc/openerp-server7.conf:

[options]
# Basic daemon configuration options ##################
root_path = /usr/local/lib/python2.7/site-packages/openerp-server/
addons_path = /usr/local/lib/python2.7/site-packages/openerp-7.0_20130917_231058-py2.7.egg/openerp/addons
# netinterface =
# interface =
port = 8069
netport = 8070
debug_mode = False
stop_after_init = False
soap = False
xmlrpc = True
netrpc = True
secure = False
cache_timeout = 100000
pidfile = /var/run/openerp/openerp-server7.pid
reportgz = False
admin_passwd = admin
login_message = False
price_accuracy = 2
csv_internal_sep = ,
translate_modules = ['all']

# Logging options #####################################
syslog = False
log_level = info
logfile = /var/log/openerp/openerp-server7.log
assert_exit_level = warn

# Database options ####################################
db_name = openerp
db_user = openerp
db_password = openerppassword
db_host = localhost
db_maxconn = 64
pg_path = /usr/local/bin
list_db = True
# import_partial =

# SMTP options ########################################
smtp_server = localhost
smtp_port = 25
smtp_user = False
smtp_password = False
email_from = False

# Use demo files? #####################################
# without_demo = True
# demo = {}

/usr/local/etc/openerp-web7.conf:

# Example configuration file for OpenERP-web
#
# This is an example configuration file, just copy it
# to openerp-web.conf and edit it to suit your needs.

# OpenERP Web server
[global]

# the ip and port the web server will be listening on
server.socket_host = "0.0.0.0" # 0.0.0.0 means all ip addresses
server.socket_port = 8180

# Sets the number of threads the server uses
server.thread_pool = 10

server.environment = "development"

# Simple code profiling
server.profile_on = False
server.profile_dir = "profile"

# Set to True if you are deploying your App behind a proxy
# e.g. Apache using mod_proxy
#tools.proxy.on = True

# If your proxy does not add the X-Forwarded-Host header, set
# the following to the *public* host url.
#tools.proxy.base = 'http://mydomain.com'

# logging
log.access_file = "/var/log/openerp-web/access7.log"
log.error_file = "/var/log/openerp-web/error7.log"

# OpenERP Server information
[openerp]
host = 'localhost'
port = '8170'
protocol = 'socket'

# Web client settings
[openerp-web]
# filter dblists based on url pattern?
# NONE: No Filter
# EXACT: Exact Hostname
# UNDERSCORE: Hostname_
# BOTH: Exact Hostname or Hostname_

dblist.filter = 'NONE'

# whether to show Databases button on Login screen or not
dbbutton.visible = True

# will be applied on company logo
company.url = ''

# options to limit data rows in M2M/O2M lists, will be overriden
# with limit="5", min_rows="5" attributes in the tree view definitions
child.listgrid.limit = 5
child.listgrid.min_rows = 5

Create the RC scripts

/usr/local/etc/rc.d/openerp-server7:

#!/bin/sh
#
# PROVIDE: openerpd7
# REQUIRE: postgresql LOGIN
#
# Add the following lines to /etc/rc.conf to enable openerp-server
#
#
# openerpd7_enable (bool): Set to "NO" by default,
#                         Set it to "YES" to enable openerp-server
#
# openerpd7_config (str):  The path to the openerp-server configuration file
#                         (defaults to /usr/local/etc/openerp-server.conf)
#
# openerpd7_flags (str):   Extra arguments to be used when invoking
#                         the openerp-server daemon.
#
# Patch submitted by (c) Franck Porcher, Ph.D, to fix some issues regarding
# pidfile, log files, and discrepencies with the main configuration file.

#  getval varname file [default_value] [separator_char]
#  - Discard comment lines (any text leading with blanks then #)
#  - Retain only the last value set


. /etc/rc.subr

name=openerpd7
command=/usr/local/bin/openerp-server7
rcvar=openerpd7_enable

load_rc_config $name

openerpd7_enable="${openerpd7_enable-"NO"}"
openerpd7_config="${openerpd7_config-"/usr/local/etc/openerp-server7.conf"}"
openerpd7_user="${openerpd7_user-"openerpd"}"
openerpd7_pidfile="${openerpd7_pidfile:-"$(grep pidfile /usr/local/etc/openerp-server7.conf | awk -F "=" ' { print $2 } ' |sed 's/[ ]//g' )"}"
openerpd7_logdir="${openerpd7_logdir:-"$(dirname `grep logfile /usr/local/etc/openerp-server7.conf | awk -F "=" ' { print $2 } ' `)"}"
openerpd7_flags="${openerpd7_flags:-"--config=${openerpd7_config}"}"

# /etc/rc.subr use $pidfile (not ${name}_pidfile)
pidfile="${openerpd7_pidfile}"

required_files="${openerpd7_config}"

start_precmd="${name}_prestart"
stop_cmd="${name}_stop"
status_cmd="${name}_status"
getval_cmd="${name}_getval"

openerpd7_prestart()
{
    local d

    d="$(dirname "${openerpd7_pidfile}")"
    if [ ! -d "${d}" ]
    then
        mkdir -p "${d}"
    fi
    chown "${openerpd7_user}" "${d}"

    d="$openerpd7_logdir"
    if [ ! -d "${d}" ]
    then
        mkdir -p "${d}"
    fi
    chown "${openerpd7_user}" "${d}"
}

openerpd7_stop()
{
    # echo "Looking for (openerp-server ${openerpd7_flags})"
    openerpd7_pid=$(pgrep -f "openerp-server ${openerpd7_flags}")
    # Try its best to stop the service
    if [ -f "${openerpd7_pidfile}" ] && [ "${openerpd7_pidfile}" = "${openerpd7_pid}" ]
    then
        echo "Stopping ${name}."
        kill -15 "$(cat "${openerpd7_pidfile}")"
    else
        if [ -n "${openerpd7_pid}" ]
        then
            echo "Stopping ${name}."
            kill -15 "${openerpd7_pid}"
        else
            echo "${name} not running? (pidfile not found)"
        fi
    fi
}

openerpd7_status()
{
    # Try its best to find the service's status
    if [ -f "${openerpd7_pidfile}" ]
    then
        openerpd7_pid="$(cat "${openerpd7_pidfile}")"
    fi

    if [ -z "${openerpd7_pid}" ]
    then
    openerpd7_pid=$(pgrep -f "openerp-server.py ${openerpd7_flags}")
    [ -n "${openerpd7_pid}" ] && echo "${openerpd7_pid}" > "${openerpd7_pidfile}"
    fi

    if [ -n "${openerpd7_pid}" ]
    then
        echo "${name} running with pid: $openerpd7_pid"
    else
        echo "${name} not running? (pid not found)"
    fi
}


command_args=" >/dev/null 2>&1 &"

load_rc_config $name
run_rc_command "$1"

/etc/rc.conf:

openerpd7_enable="YES"
openerpweb7_enable="YES"
openerpweb7_user="openerpd"

/usr/local/bin/openerp-server7:

#!/usr/local/bin/python
# EASY-INSTALL-SCRIPT: 'openerp==7.0-20130917-231058','openerp-server'
import os
os.environ['PYTHON_EGG_CACHE'] = '/tmp/python_egg_cache'
__requires__ = 'openerp==7.0-20130917-231058'
import pkg_resources
pkg_resources.run_script('openerp==7.0-20130917-231058', 'openerp-server')

Start the server

/usr/local/etc/rc.d/openerp-server7 start

If the server does not start, try to start it manually and look for any output:

su -m openerpd -c 'sh -c "/usr/local/bin/openerp-server7 --config=/usr/local/etc/openerp-server7.conf"'

How to download only one Android source code project using repo

Image

Situation

You want to submit a small patch to an Android source project.

You don't want to download the entire 14GB Android source code.

What you did

Initialize repo.

repo init -u https://android.googlesource.com/platform/manifest
Get https://gerrit.googlesource.com/git-repo
remote: Counting objects: 117, done
remote: Finding sources: 100% (117/117)
remote: Total 2879 (delta 1523), reused 2879 (delta 1523)
Receiving objects: 100% (2879/2879), 2.44 MiB | 696 KiB/s, done.
Resolving deltas: 100% (1523/1523), done.
From https://gerrit.googlesource.com/git-repo
 * [new branch]      maint      -> origin/maint
 * [new branch]      master     -> origin/master
 * [new branch]      stable     -> origin/stable
 * [new tag]         v1.0       -> v1.0
 * [new tag]         v1.0.1     -> v1.0.1
[...]
Get https://android.googlesource.com/platform/manifest
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404 Not Found
Server does not provide clone.bundle; ignoring.
remote: Counting objects: 110, done
remote: Finding sources: 100% (110/110)
remote: Total 1345 (delta 288), reused 1345 (delta 288)
Receiving objects: 100% (1345/1345), 1.13 MiB | 479 KiB/s, done.
Resolving deltas: 100% (288/288), done.
From https://android.googlesource.com/platform/manifest
[...]
 * [new tag]         android-sdk-support_r11 -> android-sdk-support_r11

Your Name  [U-dl_muriers_em\ZWM]: Erwan Martin
Your Email [ZWM@dl_muriers_em.(none)]: public@fzwte.net

Your identity is: Erwan Martin <public@fzwte.net>
is this correct [y/N]? y

repo has been initialized in /home/ZWM

Sync the project.

repo sync platform/tools/base
Traceback (most recent call last):
  File "/home/ZWM/.repo/repo/main.py", line 500, in <module>
    _Main(sys.argv[1:])
  File "/home/ZWM/.repo/repo/main.py", line 476, in _Main
    result = repo._Run(argv) or 0
  File "/home/ZWM/.repo/repo/main.py", line 155, in _Run
    result = cmd.Execute(copts, cargs)
  File "/home/ZWM/.repo/repo/subcmds/sync.py", line 624, in Execute
    submodules_ok=opt.fetch_submodules)
  File "/home/ZWM/.repo/repo/command.py", line 184, in GetProjects
    raise InvalidProjectGroupsError(arg)
error.InvalidProjectGroupsError

InvalidProjectGroupsError? Whaaaaaat!?

Why did that happen

Let's open .repo/repo/command.py and go where the command failed.

        for project in projects:
          if not missing_ok and not project.Exists:
            raise NoSuchProjectError(arg)
          if not project.MatchesGroups(groups):
            raise InvalidProjectGroupsError(arg)

Apparently, a notion of group exists. Let's open the manifest and see in what group our project is.

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
  [...]
  <project path="tools/base" name="platform/tools/base" groups="notdefault,tools" />
</manifest>

notdefault? That doesn't sound good.

What you should have done

Init repo including group tools.

repo init -u https://android.googlesource.com/platform/manifest -g tools
[...]

Sync the project.

repo sync platform/tools/base
Fetching project platform/tools/base
[...]
Fetching projects: 100% (1/1), done.
Checking out files: 100% (5614/5614), done.

Create and checkout your branch.

repo start your-local-branch-name platform/tools/base

Work your magic and upload your changes.

repo upload

Reference

How to build Android APKs on a FreeBSD server

Image The FreeBSD logo

Situation

Your team is creating Android apps. Your CI server is running FreeBSD and you'd like Jenkins or something else to be able to build your apps to have nice nightly builds.

You don't care about having adb and you especially don't want to install anything related to X11.

Example

In this example, we'll be building an app using gradle.

Install the build requirements

Install gradle

cd /usr/ports/devel/gradle && make install clean

Install the linux compatibility layer

kldload linux
cd /usr/ports/emulators/linux_base-f10 && make install distclean

Remember, you might need to use brandelf on some parts of the SDK later if they won't execute.

Install bash and link /bin/bash to /usr/local/bin/bash.

cd /usr/ports/shells/bash && make install clean
ln -s /usr/local/bin/bash /bin/bash

Install your favorite flavour of Java.

cd /usr/ports/java/openjdk7 && make install clean

Download and update the Android SDK

Download and extract the Android SDK for Linux.

fetch 'http://dl.google.com/android/android-sdk_r23-linux.tgz'
tar xzf android-sdk_r23-linux.tgz

Go into the tools folder and list the contents of the SDK repositories.

./android list sdk -u
Refresh Sources:
  Fetching https://dl-ssl.google.com/android/repository/addons_list-2.xml
  Validate XML
  Parse XML
  Fetched Add-ons List successfully
  Refresh Sources
  Fetching URL: https://dl-ssl.google.com/android/repository/repository-10.xml
  Validate XML: https://dl-ssl.google.com/android/repository/repository-10.xml
  Parse XML:    https://dl-ssl.google.com/android/repository/repository-10.xml
  Fetching URL: https://dl-ssl.google.com/android/repository/addon.xml
  Validate XML: https://dl-ssl.google.com/android/repository/addon.xml
  Parse XML:    https://dl-ssl.google.com/android/repository/addon.xml
  Fetching URL: https://dl-ssl.google.com/android/repository/addon-6.xml
  Validate XML: https://dl-ssl.google.com/android/repository/addon-6.xml
  Parse XML:    https://dl-ssl.google.com/android/repository/addon-6.xml
  Fetching URL: https://dl-ssl.google.com/glass/gdk/addon.xml
  Validate XML: https://dl-ssl.google.com/glass/gdk/addon.xml
  Parse XML:    https://dl-ssl.google.com/glass/gdk/addon.xml
  Fetching URL: https://dl-ssl.google.com/android/repository/extras/intel/addon.xml
  Validate XML: https://dl-ssl.google.com/android/repository/extras/intel/addon.xml
  Parse XML:    https://dl-ssl.google.com/android/repository/extras/intel/addon.xml
  Fetching URL: https://dl-ssl.google.com/android/repository/sys-img/android/sys-img.xml
  Validate XML: https://dl-ssl.google.com/android/repository/sys-img/android/sys-img.xml
  Parse XML:    https://dl-ssl.google.com/android/repository/sys-img/android/sys-img.xml
  Fetching URL: https://dl-ssl.google.com/android/repository/sys-img/android-wear/sys-img.xml
  Validate XML: https://dl-ssl.google.com/android/repository/sys-img/android-wear/sys-img.xml
  Parse XML:    https://dl-ssl.google.com/android/repository/sys-img/android-wear/sys-img.xml
  Fetching URL: https://dl-ssl.google.com/android/repository/sys-img/android-tv/sys-img.xml
  Validate XML: https://dl-ssl.google.com/android/repository/sys-img/android-tv/sys-img.xml
  Parse XML:    https://dl-ssl.google.com/android/repository/sys-img/android-tv/sys-img.xml
  Fetching URL: https://dl-ssl.google.com/android/repository/sys-img/x86/addon-x86.xml
  Validate XML: https://dl-ssl.google.com/android/repository/sys-img/x86/addon-x86.xml
  Parse XML:    https://dl-ssl.google.com/android/repository/sys-img/x86/addon-x86.xml
Packages available for installation or update: 63
   1- Android SDK Tools, revision 23.0.1
   2- Android SDK Platform-tools, revision 20
   3- Android SDK Build-tools, revision 20
   4- Documentation for Android 'L' Preview SDK, revision 1
   5- SDK Platform Android 4.4W, API 20, revision 1
   6- SDK Platform Android L Preview, revision 1
   7- SDK Platform Android 4.4.2, API 19, revision 3
   8- SDK Platform Android 4.3, API 18, revision 2
   9- SDK Platform Android 4.2.2, API 17, revision 2
  10- SDK Platform Android 4.1.2, API 16, revision 4
  11- SDK Platform Android 4.0.3, API 15, revision 3
  12- SDK Platform Android 4.0, API 14, revision 3
  13- SDK Platform Android 3.2, API 13, revision 1
  14- SDK Platform Android 3.1, API 12, revision 3
  15- SDK Platform Android 3.0, API 11, revision 2
  16- SDK Platform Android 2.3.3, API 10, revision 2
  17- SDK Platform Android 2.2, API 8, revision 3
  18- SDK Platform Android 2.1, API 7, revision 3
  19- SDK Platform Android 1.6, API 4, revision 3
  20- SDK Platform Android 1.5, API 3, revision 4
  21- Samples for SDK API 20, revision 1
  22- Samples for SDK API L Preview, revision 1
  23- Samples for SDK API 19, revision 5
  24- Samples for SDK API 18, revision 1
  25- Samples for SDK API 17, revision 1
  26- Samples for SDK API 16, revision 1
  27- Samples for SDK API 15, revision 2
  28- Samples for SDK API 14, revision 2
  29- Samples for SDK API 13, revision 1
  30- Samples for SDK API 12, revision 1
  31- Samples for SDK API 11, revision 1
  32- Samples for SDK API 10, revision 1
  33- Samples for SDK API 8, revision 1
  34- Samples for SDK API 7, revision 1
  35- Google APIs (x86 System Image), Android API 19, revision 5
  36- Google APIs (ARM System Image), Android API 19, revision 5
  37- Glass Development Kit Preview, Android API 19, revision 8
  38- Google APIs, Android API 18, revision 3
  39- Google APIs, Android API 17, revision 3
  40- Google APIs, Android API 16, revision 3
  41- Google APIs, Android API 15, revision 2
  42- Google APIs, Android API 14, revision 2
  43- Google APIs, Android API 13, revision 1
  44- Google TV Addon, Android API 13, revision 1
  45- Google APIs, Android API 12, revision 1
  46- Google TV Addon, Android API 12, revision 2
  47- Google APIs, Android API 11, revision 1
  48- Google APIs, Android API 10, revision 2
  49- Google APIs, Android API 8, revision 2
  50- Google APIs, Android API 7, revision 1
  51- Google APIs, Android API 4, revision 2
  52- Google APIs, Android API 3, revision 3
  53- Android Support Repository, revision 6
  54- Android Support Library, revision 20
  55- Google Play services for Froyo, revision 12
  56- Google Play services, revision 17
  57- Google Repository, revision 8
  58- Google Play APK Expansion Library, revision 3
  59- Google Play Billing Library, revision 5
  60- Google Play Licensing Library, revision 2
  61- Google USB Driver, revision 10
  62- Google Web Driver, revision 2
  63- Intel x86 Emulator Accelerator (HAXM installer), revision 4

-u means "no gui".

Install what you need.

./android update sdk --filter 1,2,3,5,6,53,54,56 -u
[license stuff]
Installing Archives:
  Preparing to install archives
  Downloading Android SDK Platform-tools, revision 20
  Installing Android SDK Platform-tools, revision 20
  Stopping ADB server failed (code -1).
    Installed Android SDK Platform-tools, revision 2099%)
  Downloading Android SDK Build-tools, revision 20
  Installing Android SDK Build-tools, revision 20
    Installed Android SDK Build-tools, revision 2099%)
  Downloading SDK Platform Android 4.4W, API 20, revision 1
  Installing SDK Platform Android 4.4W, API 20, revision 1
    Installed SDK Platform Android 4.4W, API 20, revision 197%)
  Downloading SDK Platform Android L Preview, revision 1
  Installing SDK Platform Android L Preview, revision 1
    Installed SDK Platform Android L Preview, revision 197%)
  Downloading Android Support Repository, revision 6
  Installing Android Support Repository, revision 6
    Installed Android Support Repository, revision 699%)
  Downloading Android Support Library, revision 20
  Installing Android Support Library, revision 20
    Installed Android Support Library, revision 2093%)
  Downloading Google Play services, revision 17
  Installing Google Play services, revision 17
    Installed Google Play services, revision 1796%)
  Downloading Android SDK Tools, revision 23.0.1
  Installing Android SDK Tools, revision 23.0.1
    Installed Android SDK Tools, revision 23.0.199%)
  Stopping ADB server failed (code -1).
  Unable to run 'adb': Cannot run program "/root/android-sdk-linux/platform-tools/adb": error=2, No such file or directory.
  Starting ADB server failed (code -1).
null  Done. 8 packages installed.

Build your apps!

Edit local.properties and set your SDK location. Alternatively, you can set the environment variable ANDROID_HOME.

sdk.dir=/some/where/android-sdk-linux

Start gradle and see if it complains.

gradle
[some stuff]
Welcome to Gradle 1.12.

To run a build, run gradle <task> ...

To see a list of available tasks, run gradle tasks

To see a list of command-line options, run gradle --help

BUILD SUCCESSFUL

Total time: 14.139 secs

If not, build!

gradle assembleRelease
[many things]
14:25:54.743 [LIFECYCLE] [org.gradle.BuildResultLogger]
14:25:54.744 [LIFECYCLE] [org.gradle.BuildResultLogger] BUILD SUCCESSFUL
14:25:54.746 [LIFECYCLE] [org.gradle.BuildResultLogger]
14:25:54.748 [LIFECYCLE] [org.gradle.BuildResultLogger] Total time: 1 mins 56.878 secs
ll Truc/build/outputs/apk/
-rw-r--r--  1 root  wheel  1064222 Jul  1 14:25 Truc-release-unaligned.apk
-rw-r--r--  1 root  wheel  1064495 Jul  1 14:25 Truc-release.apk

You can also use the SDK Manager Plugin for gradle, so that your SDK is updated automatically when you build your project.

Problems I met

gradle won't start

gradle

FAILURE: Build failed with an exception.

* What went wrong:
net.rubygrapefruit.platform.internal.jni.PosixFileFunctions.symlink(Ljava/lang/String;Ljava/lang/String;Lnet/rubygrapefruit/platform/internal/FunctionResult;)V

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

libstdc++.so.6 is missing. Install gcc.

cd /usr/ports/lang/gcc && make install clean

Ambiguous method overloading for method java.io.File

> Ambiguous method overloading for method java.io.File#<init>.
  Cannot resolve which method to invoke for [null, class java.lang.String] due to overlapping prototypes between:
        [class java.lang.String, class java.lang.String]
        [class java.io.File, class java.lang.String]

gradle couldn't find the SDK. Did you set sdk.dir in local.properties?


The SDK download packages for Windows instead of Linux

The SDK doesn't recognize FreeBSD, so it downloads packages for Windows. It needs to be patched.

Download the sdk source.

git clone https://android.googlesource.com/platform/tools/base

Apply the following patch to the source:

diff -Naur A/common/src/main/java/com/android/SdkConstants.java B/common/src/main/java/com/android/SdkConstants.java
--- A/common/src/main/java/com/android/SdkConstants.java        2014-07-01 16:44:55.000000000 +0200
+++ B/common/src/main/java/com/android/SdkConstants.java        2014-07-01 19:57:58.000000000 +0200
@@ -596,6 +596,8 @@
             return PLATFORM_WINDOWS;
         } else if (os.startsWith("Linux")) {                //$NON-NLS-1$
             return PLATFORM_LINUX;
+        } else if (os.startsWith("FreeBSD")) {              //$NON-NLS-1$
+            return PLATFORM_LINUX;
         }

         return PLATFORM_UNKNOWN;
diff -Naur A/sdklib/src/main/java/com/android/sdklib/internal/repository/archives/ArchFilter.java B/sdklib/src/main/java/com/android/sdklib/internal/repository/archives/ArchFilter.java
--- A/sdklib/src/main/java/com/android/sdklib/internal/repository/archives/ArchFilter.java      2014-07-01 19:57:30.000000000 +0200
+++ B/sdklib/src/main/java/com/android/sdklib/internal/repository/archives/ArchFilter.java      2014-07-01 19:55:33.000000000 +0200
@@ -210,6 +210,8 @@
             hostOS = HostOs.WINDOWS;
         } else if (os.startsWith("Linux")) {                //$NON-NLS-1$
             hostOS = HostOs.LINUX;
+        } else if (os.startsWith("FreeBSD")) {              //$NON-NLS-1$
+            hostOS = HostOs.LINUX;
         }

         BitSize jvmBits;

Rebuild the patched files.

javac sdklib/src/main/java/com/android/sdklib/internal/repository/archives/ArchFilter.java -cp "sdklib/src/main/java:common/src/main/java"
javac common/src/main/java/com/android/SdkConstants.java

Replace the files inside the jar.

cd sdklib/src/main/java/ && jar uf /some/where/android-sdk-linux/tools/lib/sdklib.jar com/android/sdklib/internal/repository/archives/ArchFilter.class
cd common/src/main/java && jar uf /some/where/android-sdk-linux/tools/lib/common.jar com/android/SdkConstants.class

See this patch on the Android website: https://android-review.googlesource.com/#/c/100271/.

Profit!/code

How to run a laravel package's migrations

Problem

Let's say you fetched a laravel package using composer and whatever, and you want to run its migration files (to have it create its tables for instance).

# php artisan migrate
Nothing to migrate.

Hum, laravel won't find anything to migrate.

Solution

The trick is to give artisan the path containing the migration files:

# php artisan migrate --path=vendor/vendorname/package-name/src/migrations
Migrated: 2013_11_21_000001_create_something
Migrated: 2013_11_21_000002_create_something_else
Migrated: 2014_03_21_000003_add_stuff

How to sign POST requests with OAuth 1.0 using oauth-signpost and HttpURLConnection

Situation

Signpost's documentation states that the library cannot be used along with HttpURLConnection to sign standard POST requests. This is not entirely true.

Method setAdditionalParameters() can be used to add a few parameters to the OAuth base string. The trick is to add them already percent-encoded.

Code sample: signing standard URL-encoded POST requests

package test_oauth;

import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStreamWriter;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.Iterator;
import java.util.Scanner;

import oauth.signpost.OAuth;
import oauth.signpost.OAuthConsumer;
import oauth.signpost.basic.DefaultOAuthConsumer;
import oauth.signpost.exception.OAuthCommunicationException;
import oauth.signpost.exception.OAuthExpectationFailedException;
import oauth.signpost.exception.OAuthMessageSignerException;
import oauth.signpost.http.HttpParameters;

public class TestOAuth {

	public static void main(String[] args) throws IOException, OAuthMessageSignerException, OAuthExpectationFailedException, OAuthCommunicationException {
		final String endpointUrl = "http://some-api.example.org/index.php/api/v1/end/point";

		//Create an HttpURLConnection and add some headers
		URL url = new URL(endpointUrl);
		HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
		urlConnection.setRequestProperty("Accept", "application/json");
		urlConnection.setRequestMethod("POST");
		urlConnection.setDoOutput(true);

		//Build the list of parameters
		HttpParameters params =  new HttpParameters();
		params.put("param1", "49607,49608,49609");
		params.put("param2", "1396430420");
		params.put("param3", "coucou");
		
		//Sign the request
		//The key to signing the POST fields is to add them as additional parameters, but already percent-encoded; and also to add the realm header.
		OAuthConsumer dealabsConsumer = new DefaultOAuthConsumer ("5314c6fbe7437", "5314c6fbe7bfd");
		HttpParameters doubleEncodedParams =  new HttpParameters();
        Iterator<String> iter = params.keySet().iterator();
        while (iter.hasNext()) {
            String key = iter.next();
            doubleEncodedParams.put(key, OAuth.percentEncode(params.getFirst(key)));
        }
        doubleEncodedParams.put("realm", endpointUrl);
		dealabsConsumer.setAdditionalParameters(doubleEncodedParams);
        dealabsConsumer.sign(urlConnection);
        
        //Create the POST payload
        StringBuilder sb = new StringBuilder();
        iter = params.keySet().iterator();
        for (int i = 0; iter.hasNext(); i++) {
            String param = iter.next();
            if (i > 0) {
                sb.append("&");
            }
            sb.append(params.getAsQueryString(param, false));  
        }
        
        //Send the payload to the connection
        String formEncoded = sb.toString();;
		OutputStreamWriter outputStreamWriter = null;
        try {
            outputStreamWriter = new OutputStreamWriter(urlConnection.getOutputStream(), "UTF-8");
            outputStreamWriter.write(formEncoded);
        } finally {
            if (outputStreamWriter != null) {
                outputStreamWriter.close();
            }
        }
        
        //Send the request and read the output
		try {
			System.out.println("Response: " + urlConnection.getResponseCode() + " " + urlConnection.getResponseMessage());
			InputStream in = new BufferedInputStream(urlConnection.getInputStream());
			String inputStreamString = new Scanner(in,"UTF-8").useDelimiter("\\A").next();
			System.out.println(inputStreamString);
		}
		finally {
			urlConnection.disconnect();
		}
	}

}

This code was tested on Java 7 on a PC, and on Android.

Code sample: signing a request using a raw JSON (or anything else) payload

Added on 2014-09-10. Asked by someone by email.

package test_oauth;
 
import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStreamWriter;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.Scanner;
 
import oauth.signpost.OAuthConsumer;
import oauth.signpost.basic.DefaultOAuthConsumer;
import oauth.signpost.exception.OAuthCommunicationException;
import oauth.signpost.exception.OAuthExpectationFailedException;
import oauth.signpost.exception.OAuthMessageSignerException;
import oauth.signpost.http.HttpParameters;
 
public class TestOAuth {
 
    public static void main(String[] args) throws IOException, OAuthMessageSignerException, OAuthExpectationFailedException, OAuthCommunicationException {
        final String endpointUrl = "http://some-api.example.org//api/v1/give/me/some/json";
 
        final String JSONPayload = "{\"this\": \"is\", \"some\": \"JSON\"}";
        
        //Create an HttpURLConnection and add some headers
        URL url = new URL(endpointUrl);
        HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
        urlConnection.setRequestProperty("Accept", "application/json");
        urlConnection.setRequestProperty("Content-Type", "application/json; charset=utf8");
        urlConnection.setRequestMethod("POST");
        urlConnection.setDoOutput(true);
        
        //Sign the request
        OAuthConsumer dealabsConsumer = new DefaultOAuthConsumer ("5314c6fbe7437", "5314c6fbe7bfd");
        HttpParameters doubleEncodedParams =  new HttpParameters();
        doubleEncodedParams.put("realm", endpointUrl);
        dealabsConsumer.setAdditionalParameters(doubleEncodedParams);
        dealabsConsumer.sign(urlConnection);

        //Send the payload to the connection
        OutputStreamWriter outputStreamWriter = null;
        try {
            outputStreamWriter = new OutputStreamWriter(urlConnection.getOutputStream(), "UTF-8");
            outputStreamWriter.write(JSONPayload);
        } finally {
            if (outputStreamWriter != null) {
                outputStreamWriter.close();
            }
        }
         
        //Send the request and read the output
        try {
            System.out.println("Response: " + urlConnection.getResponseCode() + " " + urlConnection.getResponseMessage());
            InputStream in = new BufferedInputStream(urlConnection.getInputStream());
            String inputStreamString = new Scanner(in,"UTF-8").useDelimiter("\\A").next();
            System.out.println(inputStreamString);
        }
        finally {
            urlConnection.disconnect();
        }
    }
 
}

How to have cpuminer use all CPUs on FreeBSD

Image

Situation

You're trying to mine some cryptocoins on your FreeBSD box. Unfortunately, the mining process seems to occur on only one CPU, despite cpuminer creating and binding threads.

[2014-03-23 01:41:53] 8 miner threads started, using 'X11' algorithm.
[2014-03-23 01:41:53] Starting Stratum on stratum+tcp://drkpool.com:3333
[2014-03-23 01:41:53] Binding thread 0 to cpu 0
[2014-03-23 01:41:53] Binding thread 1 to cpu 1
[2014-03-23 01:41:53] Binding thread 2 to cpu 2
[2014-03-23 01:41:53] Binding thread 3 to cpu 3
[2014-03-23 01:41:53] Binding thread 4 to cpu 4
[2014-03-23 01:41:53] Binding thread 5 to cpu 5
[2014-03-23 01:41:53] Binding thread 6 to cpu 6
[2014-03-23 01:41:53] Binding thread 7 to cpu 7
[2014-03-23 01:41:56] Stratum detected new block

Using ktrace yields some information. The thread-cpu binding is failing.

 23429 minerd   CALL  write(0x2,0x7fffff5fa660,0x30)
 23429 minerd   GIO   fd 2 wrote 48 bytes
       "[2014-03-23 00:45:32] Binding thread 1 to cpu 1
       "
 23429 minerd   RET   write 48/0x30
 23429 minerd   CALL  _umtx_op(0x803019060,0x16,0,0,0)
 23429 minerd   RET   _umtx_op 0
 23429 minerd   CALL  cpuset_setaffinity(0x3,0x3,0xffffffffffffffff,0x10,0x7fffff5faf70)
 23429 minerd   RET   cpuset_setaffinity -1 errno 11 Resource deadlock avoided
 23429 minerd   CALL  nanosleep(0x7fffff5fad90,0x7fffff5fad80)
 23429 minerd   RET   _umtx_op 0
 23429 minerd   CALL  mmap(0,0x400000,0x3<PROT_READ|PROT_WRITE>,0x1002<MAP_PRIVATE|MAP_ANON>,0xffffffff,0)
 23429 minerd   RET   mmap 34431041536/0x804400000
 23429 minerd   CALL  _umtx_op(0x803019060,0x16,0,0,0)
 23429 minerd   RET   _umtx_op 0

Solution

Use cpuset to change the affinity of the process once it's launched.

cpuset -c -l 0-15 -p 3044

Adapt the command to your number of CPUs and the PID of your miner process.

See kern/163585 for more information.

How to connect to a MySQL/MariaDB server with PHP using the dialog plugin.

Situation

You want to authenticate your MySQL/MariaDB users using an authentication plugin (for instance: pam). This kind of auth uses the dialog plugin. You can login using the mysql command line client and using Perl's DBI, but not without PHP.

Fatal error: Uncaught exception 'PDOException' with message 'SQLSTATE[HY000] [2054] The server requested authentication method unknown to the client'

Solution

PHP now uses its own MySQL driver (mysqlnd), instead of the good old libmysqlclient.

The solution is to rebuild PHP using the right configure options.

                --enable-pdo \
                --with-mysqli=/usr/local/bin/mysql_config \
                --with-pdo-mysql=/usr/local/bin/mysql_config \

Reference: http://dev.mysql.com/doc/apis-php/en/apis-php-mysqli.installation.linux.html

How to use Wireshark on Windows to watch a remote UNIX system, using SSH

Wireshark&#039;s pipe interface

Situation

You're using tcpdump on a UNIX box but you would like to use Wireshark for all its awesome features (gui, analysis, etc.) on your local Windows desktop computer.

Solution

Connect to your box using SSH and pipe the output of tcpdump into wireshark.

"c:\Program Files (x86)\PuTTY\plink.exe" -ssh root@my-unix-box.example.com "tcpdump -w - -s 65535 [tcpdump condition] " | "c:\Program Files\Wireshark\wireshark.exe" -i -

plink is bundled with putty, but you can download it separatly.

How to get rid of "Warning: using insecure memory" using spamassasin on FreeBSD

Situation

You run sa-update periodically and in your logs you get some lousy:

Warning: using insecure memory! 

Fix

The well known fix is to add some option into the gpg configuration file.

The key is to know where to find it:

/usr/local/etc/mail/spamassassin/sa-update-keys/gpg.conf

no-secmem-warning

How to create a rar file with every file matching a pattern (including subdirectories) on Windows

Creating the rar file

How to create a rar file with any file matching a pattern (including subdirectories) on Windows.

forfiles /s /m wallet.dat /c "cmd /c echo @relpath" | "c:\Program Files\WinRAR\Rar.exe" a -dh -rr524288 wallets.rar @

RAR 4.20   Copyright (c) 1993-2012 Alexander Roshal   9 Jun 2012
Registered to zewaren

Updating archive wallets.rar

Updating  .\Alphacoin\wallet.dat                                      OK
Updating  .\AmericanCoin\wallet.dat                                   OK
Updating  .\AndroidsToken\wallet.dat                                  OK
Updating  .\Betacoin\wallet.dat                                       OK
Updating  .\Bitcoin\wallet.dat                                        OK
Updating  .\Colossuscoin\wallet.dat                                   OK
Updating  .\digitalcoin\wallet.dat                                    OK
Updating  .\eMark\wallet.dat                                          OK
Updating  .\Fastcoin\wallet.dat                                       OK
Updating  .\Feathercoin\wallet.dat                                    OK
Updating  .\Florincoin\wallet.dat                                     OK
Updating  .\Franko\wallet.dat                                         OK
Updating  .\Gridcoin\wallet.dat                                       OK
Updating  .\IncaKoin\wallet.dat                                       OK
Updating  .\Litecoin\wallet.dat                                       OK
Updating  .\Luckycoin\wallet.dat                                      OK
Updating  .\Namecoin\wallet.dat                                       OK
Updating  .\NetCoin\wallet.dat                                        OK
Updating  .\NovaCoin\wallet.dat                                       OK
Updating  .\Ocoin\wallet.dat                                          OK
Updating  .\Phoenixcoin\wallet.dat                                    OK
Updating  .\PPCoin\wallet.dat                                         OK
Updating  .\Primecoin\wallet.dat                                      OK
Updating  .\Protoshares\wallet.dat                                    OK
Updating  .\Redcoin\wallet.dat                                        OK
Updating  .\RoyalCoin\wallet.dat                                      OK
Updating  .\Terracoin\wallet.dat                                      OK
Updating  .\Worldcoin\wallet.dat                                      OK
Updating  .\YaCoin\wallet.dat                                         OK
Updating  .\Sexcoin\wallet.dat                                        OK
Done

This example will find any file named wallet.dat and include it in the rar file. You can use wildchars in the pattern.

This will preserve the directory structure.

-dh allows you to add files that are in use by another process.

-rr524288 adds a recovery record to the archive, with the maximum number of sectors (never too cautious).

Checking the rar file

"c:\Program Files\WinRAR\Rar.exe" vb wallets.rar
Alphacoin\wallet.dat
AmericanCoin\wallet.dat
AndroidsToken\wallet.dat
Betacoin\wallet.dat
Bitcoin\wallet.dat
Colossuscoin\wallet.dat
digitalcoin\wallet.dat
eMark\wallet.dat
Fastcoin\wallet.dat
Feathercoin\wallet.dat
Florincoin\wallet.dat
Franko\wallet.dat
Gridcoin\wallet.dat
IncaKoin\wallet.dat
Litecoin\wallet.dat
Luckycoin\wallet.dat
Namecoin\wallet.dat
NetCoin\wallet.dat
NovaCoin\wallet.dat
Ocoin\wallet.dat
Phoenixcoin\wallet.dat
PPCoin\wallet.dat
Primecoin\wallet.dat
Protoshares\wallet.dat
Redcoin\wallet.dat
RoyalCoin\wallet.dat
Terracoin\wallet.dat
Worldcoin\wallet.dat
YaCoin\wallet.dat
Sexcoin\wallet.dat

Image

Syndicate content