tag:blogger.com,1999:blog-77604252760560322232024-02-18T20:26:43.735-08:00PLABOProgramming Linux And Bioiformatics Omnibus
(by Pablo Marin-Garcia)Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.comBlogger117125tag:blogger.com,1999:blog-7760425276056032223.post-19018823999830478642019-08-30T09:59:00.001-07:002019-08-30T11:02:08.101-07:00PyCharm 2019.2 IDE was not starting on MacOSX: missing jdkPyCharm 2019.2 is looking for java8 and does not come with a jdk folder. So if you have java11 it will not even start. I have had this problem with the new PyCharm installed from the ToolBox, but also with the new manually installed<br />
<br />
<b>Solution</b>:<br />
Either install java8 or if you have a working app from JetBrains like a previous manual installation you can copy the jdk from there and paste itin your new copy.<br />
<br />
The ToolBox apps are installed in<br />
<br />
<br />
<div class="shell">
<pre>~/Library/Application Support/JetBrains/Toolbox/apps/PyCharm-P/ch-0/192.6262.63/PyCharm.app
</pre>
</div>
<br />
In order to copy the previous jdk from a working pycharm (or other JetBrains app) you can navigate to a manually installed old app in "/Applications" and right click in the app and select "Show Package Content". In the "Contents" folder there would be a "jdk" folder. Copy it to your not working JetBrains app.<br />
<br />
<br />
REFS:<br />
<a href="https://intellij-support.jetbrains.com/hc/en-us/community/posts/360003340039-PyCharm-2019-1-won-t-open-through-Toolbox-App">https://intellij-support.jetbrains.com/hc/en-us/community/posts/360003340039-PyCharm-2019-1-won-t-open-through-Toolbox-App</a><br />
<br />
<b>Still a problem:</b><br />
<br />
PyCharm2019.2 does not open the terminal (OSX 10.10.5)<br />
<b><br /></b>
<br />
<style type="text/css">
.p1 {margin: 0.0px 0.0px 0.0px 0.0px; padding: 1em; font: 10.0px Monaco; color: #f5f5f5; background-color: #000000}
.s1 {font-variant-ligatures: no-common-ligatures}
</style>
<br />
<div class="p1">
<span class="s1">Cannot open Local Terminal</span></div>
<div class="p1">
<span class="s1">Failed to start [/bin/bash, --rcfile, /Applications/PyCharm.app/Contents/plugins/terminal/jediterm-bash.in, -i]</span></div>
<b><br /></b>
<b><br /></b>Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com0tag:blogger.com,1999:blog-7760425276056032223.post-46382697916868284132017-08-15T10:33:00.001-07:002017-08-15T10:34:37.319-07:00Change Docker default software dir<h2>
Change Docker default software dir</h2>
<br />
For Docekr 17.xx in Ubuntu 16.04 <b>don't follow</b> the advice of updating the <span style="background-color: #cfe2f3;"><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">/etc/default/docker</span></span> and change <span style="background-color: #cfe2f3; font-family: "courier new" , "courier" , monospace; font-size: x-small;">-g</span> option in <span style="background-color: #cfe2f3; font-family: "courier new" , "courier" , monospace; font-size: x-small;">DOCKER_OPTS</span>. This advice is for <b>OLD debian using upstar</b> system. If <span style="background-color: yellow;">your linux uses Systemd do the following: use the </span><span style="background-color: white;"> </span><span style="background-color: #cfe2f3; color: #0c5176; font-family: "courier new" , "courier" , monospace; font-size: 13.5px; white-space: nowrap;">/etc/docker/daemon.json</span> and add a <span style="background-color: #cfe2f3;"><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>graph</b></span></span> directive<br />
<br />
<div class="box">
<pre>{
"graph": "/mnt_docker/docker_files"
}
</pre>
</div>
<br />
<br />
<br />Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com0tag:blogger.com,1999:blog-7760425276056032223.post-5918921336542505672016-10-10T12:59:00.000-07:002016-11-21T00:39:42.710-08:00XRDP in centos7: installation, enabling it and setting port for reconnecting a previous session<br />
RDP is more efficient and has better performing than VNC and probably is more secure.<br />
<br />
<h4>
Installation</h4>
The adventage of x11vnc is that you have the same desktop that is open in your workstation and two persons can share the screen. The cons is that you need to have a logged user in the Xs before using it. If you restart the
machine then you need to do some tricks to us x11vnc as root to be able
to go to the splash window.<br />
<br />
XRDP is better but by default you open different sessions each time and is not trivial how to get the current session active in your workstation.<br />
<br />
Anyway, if you use Centos7 you will have probles usig XRDP because there are some issues with selenux (the security system for applications in redhat) <br />
<br />
XRDP can be installed from EPEL repo (the essential extra packages).<br />
<div class="shell">
<pre># install EPEL repo
$ sudo yum install epel-release
$ sudo yum update
# install XRDP
$ sudo yum install xrdp tigervnc-server</pre>
<pre>$ sudo service xrdp start </pre>
<pre>$ sudo service xrdp-sesman start</pre>
</div>
<br />
After installing, activate the service
<br />
<div class="shell">
<pre> $ sudo systemctl enable xrdp.service
</pre>
</div>
Confirm that XRDP is runing
<br />
<div class="shell">
<pre># netstat -antup | grep xrdp
tcp 0 0 0.0.0.0:3389 0.0.0.0:* LISTEN 1508/xrdp
tcp 0 0 127.0.0.1:3350 0.0.0.0:* LISTEN 1507/xrdp-sesman
</pre>
</div>
But the job is not finished. By default XRDP listen to 3389 but this service is not allowed to listen to this port by default. You need to fix security and firewall to allow XRDP listen to 3389 (selenux) and allow external connections to 2289 (firewall)
<br />
<div class="shell">
<pre># allow selemux (use one of the two options. Option 2 is prefered)
## option 1
$ sudo chcon --type=bin_t /usr/sbin/xrdp
$ sudo chcon --type=bin_t /usr/sbin/xrdp-sesman
## option 2
$ sudo semanage fcontext -a -t bin_t /usr/sbin/xrdp
$ sudo semanage fcontext -a -t bin_t /usr/sbin/xrdp-sesman
$ sudo restorecon -v /usr/sbin/xrdp*
# using chcon updates the SELinux context temporarily until the next relabelling.
# A permanent method is to use semanage and restorecon
# allow firewall for external connections (just in case is not open by default)
$ sudo firewall-cmd --permanent --zone=public --add-port=3389/tcp
$ sudo firewall-cmd --reload </pre>
</div>
<h4>
Configuration</h4>
The config file is in <span class="code_inline"> /etc/xrdp/xrdp.ini </span>
<br />
The port by default is 3389 and you can change here:
<br />
<div class="shell">
<pre>[globals]
# xrdp.ini file version number
ini_version=1
bitmap_cache=yes
bitmap_compression=yes
port=3389
</pre>
</div>
<br />
And the setting for reconnecting to the previous session is in section "[XRDP1]" for port attribute. Set to ask-1
<br />
<div class="shell">
<pre>[xrdp1]
name=sesman-Xvnc
lib=libvnc.so
username=ask
password=ask
ip=127.0.0.1
port=ask-1
delay_ms=2000
</pre>
</div>
<br />
<b>ask</b> would ask you for a port in the log in. If no port given, with the <b>-1</b> a new one is assigned<br />
<br />
If you can remember the current port for your session see next section <br />
<h4>
How to find the current XRDP session port</h4>
You could login a ssh session and find out the number by
<br />
<div class="shell">
<pre>netstat -tulpn | grep vnc
</pre>
</div>
and you will get something like the following
<br />
<div class="shell">
<pre>tcp 0 0 127.0.0.1:5910 0.0.0.0:* LISTEN 5365/Xvnc
</pre>
</div>
and then you know 5910 was the port you connected to.<br />
<br />
<br />
<br />
More infor about XRDP and X11vnc and vnc links can be found at http://c-nergy.be/blog/?p=6063 Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com0tag:blogger.com,1999:blog-7760425276056032223.post-37063899856287559402016-02-13T06:17:00.001-08:002016-02-13T06:18:02.347-08:00Choosing the right dark theme in R studioRstudio Dark Themes<br />
<br />
Seems that Iddle fingers, Mervibore soft,Twilight and Tomorrow Night have the softest color schema with good contrast for comments and not to bright colours for the long coding nights ahead. I like the comments in italic and with not a prominent color (I want to keep my attention at the code) so Twilight is my current chose.<br />
<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhK1nGXw1U4C5Sa2VHPS1sP4GLE6nD9BRcnQifJ9wDDYRFdHNm_nfRLeAgqHRsp8oKqGaxZY-3nQQf8HMF864vELjRDd6Mx2S5fG54DN2TFyE70O88aG6loKd91lfy0Q7UXePMd3NJ25Z6U/s1600/Screen+Shot+2016-02-13+at+14.26.14.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhK1nGXw1U4C5Sa2VHPS1sP4GLE6nD9BRcnQifJ9wDDYRFdHNm_nfRLeAgqHRsp8oKqGaxZY-3nQQf8HMF864vELjRDd6Mx2S5fG54DN2TFyE70O88aG6loKd91lfy0Q7UXePMd3NJ25Z6U/s320/Screen+Shot+2016-02-13+at+14.26.14.png" width="277" /></a><br />
<br />
<br />
All dark themes in R-studio<br />
<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSpCkyX2y2hWKMJQcnEy-PNHA8Nz_S7sOwbWPqzlN9VYRjGix6_zJDXZosrNKf46fwcpjFzWsB7Wnj4HY0T7Bhns-JcaZO-ReOsUFsXAVxRtjgYrSoI6vDUZYIYvz207n4o2hlTwyeO4o8/s1600/Screen+Shot+2016-02-13+at+14.26.37.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSpCkyX2y2hWKMJQcnEy-PNHA8Nz_S7sOwbWPqzlN9VYRjGix6_zJDXZosrNKf46fwcpjFzWsB7Wnj4HY0T7Bhns-JcaZO-ReOsUFsXAVxRtjgYrSoI6vDUZYIYvz207n4o2hlTwyeO4o8/s320/Screen+Shot+2016-02-13+at+14.26.37.png" width="280" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkJnPvUNfVb66pjPxMcBtnSNcl1vkCszXzEXnECVRqVPP7nI8ZUndRAJsXTNAFp9flE791WVh3brNpwMACWqMEGMW7JIxJGU7i_pyzVMd5BMeHq7YCaejgFxWJMoDMbBoQoXA7C4cv2wBI/s1600/Screen+Shot+2016-02-13+at+14.26.45.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkJnPvUNfVb66pjPxMcBtnSNcl1vkCszXzEXnECVRqVPP7nI8ZUndRAJsXTNAFp9flE791WVh3brNpwMACWqMEGMW7JIxJGU7i_pyzVMd5BMeHq7YCaejgFxWJMoDMbBoQoXA7C4cv2wBI/s320/Screen+Shot+2016-02-13+at+14.26.45.png" width="281" /></a><br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBdT5IYdB01v-UgsThjaQ3yx5oOzFvuwax9bbWMbiHPMIbqLzOnr8oQcPIlXIGMVhv7DjxvAAkJdT3jERbudsd0nYTkuiUTTEjAzHN-H78ll8WVyTAFDaZPROtV3A9jg4JbnA8tofyLKWU/s1600/Screen+Shot+2016-02-13+at+14.26.54.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBdT5IYdB01v-UgsThjaQ3yx5oOzFvuwax9bbWMbiHPMIbqLzOnr8oQcPIlXIGMVhv7DjxvAAkJdT3jERbudsd0nYTkuiUTTEjAzHN-H78ll8WVyTAFDaZPROtV3A9jg4JbnA8tofyLKWU/s320/Screen+Shot+2016-02-13+at+14.26.54.png" width="276" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0X9id7D0k9BUXpmRLX-VKskCLsVVBBse8mf_6Mh1WueRP85uWmZG49WK6kKcb7Vej3zlydMk6Vf-U5rvKCSRzlol0rdu9hwx2I3CQOSZ-8q2H7zpC1L_apeKFGeYcTiNtqqFrJDM4oWR8/s1600/Screen+Shot+2016-02-13+at+14.23.55.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0X9id7D0k9BUXpmRLX-VKskCLsVVBBse8mf_6Mh1WueRP85uWmZG49WK6kKcb7Vej3zlydMk6Vf-U5rvKCSRzlol0rdu9hwx2I3CQOSZ-8q2H7zpC1L_apeKFGeYcTiNtqqFrJDM4oWR8/s320/Screen+Shot+2016-02-13+at+14.23.55.png" width="279" /></a><br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhcvnUqib3bVPU3-1z5Kst5UDqRo_VnAGNID8DhRqNcFE7hdOMZHcrOVXtyYTHGHN1Zby8ESnAYPEsQQ1nR8I0h89AQiyr1p-_qxJDNUHY1Xv1Gpl99sjJ9acGnW1aND7EpVFlbLpMH6QpV/s1600/Screen+Shot+2016-02-13+at+14.24.10.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhcvnUqib3bVPU3-1z5Kst5UDqRo_VnAGNID8DhRqNcFE7hdOMZHcrOVXtyYTHGHN1Zby8ESnAYPEsQQ1nR8I0h89AQiyr1p-_qxJDNUHY1Xv1Gpl99sjJ9acGnW1aND7EpVFlbLpMH6QpV/s320/Screen+Shot+2016-02-13+at+14.24.10.png" width="272" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPcTjU8oKEmwrjoW1VXnoPNmgbXinmeLjbb-bIlD1n4hRFAyq1p1d7lYpllZMi1N-WeVUt_QyKbiSkC9WiMxzZmDZJoVuwFOMQF2qiLUZgLH6FBRUeUsEueTESndOROhZ-FX2_bpOsCZpC/s1600/Screen+Shot+2016-02-13+at+14.24.21.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPcTjU8oKEmwrjoW1VXnoPNmgbXinmeLjbb-bIlD1n4hRFAyq1p1d7lYpllZMi1N-WeVUt_QyKbiSkC9WiMxzZmDZJoVuwFOMQF2qiLUZgLH6FBRUeUsEueTESndOROhZ-FX2_bpOsCZpC/s320/Screen+Shot+2016-02-13+at+14.24.21.png" width="276" /></a><br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEig5vffW8y8A-0ITY8QoZxj_27jAafQzWlJdb_HKYEUaANaE2DC9F91-H3Y4MGhrZL0eujFbsgeDyFWHeDI9YApyl4zWNmaJhjvaglvlD1neFdHKqyMD3sasy7M2CAimmJASWCy9dCN3bVQ/s1600/Screen+Shot+2016-02-13+at+14.24.27.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEig5vffW8y8A-0ITY8QoZxj_27jAafQzWlJdb_HKYEUaANaE2DC9F91-H3Y4MGhrZL0eujFbsgeDyFWHeDI9YApyl4zWNmaJhjvaglvlD1neFdHKqyMD3sasy7M2CAimmJASWCy9dCN3bVQ/s320/Screen+Shot+2016-02-13+at+14.24.27.png" width="276" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgM7OSHrbVuKFefRcAculnxuxcetghUf16G8NDkBTEHRnzDvame9CwIfVksRWzWx4mIg6mbghzk5vx7S2Dr9XXYtzV05ScVYv2p9rDcK9SvQ3L16x43L246Bdze4W8Q-hA27BIRLBuHP2kO/s1600/Screen+Shot+2016-02-13+at+14.24.35.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgM7OSHrbVuKFefRcAculnxuxcetghUf16G8NDkBTEHRnzDvame9CwIfVksRWzWx4mIg6mbghzk5vx7S2Dr9XXYtzV05ScVYv2p9rDcK9SvQ3L16x43L246Bdze4W8Q-hA27BIRLBuHP2kO/s320/Screen+Shot+2016-02-13+at+14.24.35.png" width="278" /></a><br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6t4X478t9WPq1CLQ1QnCPBnZsGFcx90grp9jnJWG-fJySBXbaCfsLKDmMzNcH8iBL72SsDEaT3Yvde8qfMRF13M0BpjN3PtocIdw3Iov9FNlhGT3b7TGfOlJS5frHhiX1v40cNovtxsRT/s1600/Screen+Shot+2016-02-13+at+14.24.42.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6t4X478t9WPq1CLQ1QnCPBnZsGFcx90grp9jnJWG-fJySBXbaCfsLKDmMzNcH8iBL72SsDEaT3Yvde8qfMRF13M0BpjN3PtocIdw3Iov9FNlhGT3b7TGfOlJS5frHhiX1v40cNovtxsRT/s320/Screen+Shot+2016-02-13+at+14.24.42.png" width="275" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWUFZdf2L59zjUnMdkZrG0GBAJeeHMCEqjKEGwjMPiDskhqOE4OIwtOPXEDi-PSSYbghGeYSucXvZP4MWkJxnCCjma3BEjNiW54Jqvb55yjPPRQJbbyTZQJ7LAtLeMxXt3mIEjlzg1wZI0/s1600/Screen+Shot+2016-02-13+at+14.24.49.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWUFZdf2L59zjUnMdkZrG0GBAJeeHMCEqjKEGwjMPiDskhqOE4OIwtOPXEDi-PSSYbghGeYSucXvZP4MWkJxnCCjma3BEjNiW54Jqvb55yjPPRQJbbyTZQJ7LAtLeMxXt3mIEjlzg1wZI0/s320/Screen+Shot+2016-02-13+at+14.24.49.png" width="276" /></a><br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgtj8W9jLssimyTj5NEtIaXmEz_6973tb2FeOh0aoo2UuxBuHyvqHiDIEwowoJzsLxWCpZuLTmDJNP4-oGh605pvXc_POzAwqJLprgy5JNIt_Y_5RV0zNYsuRjoZBwjwkVh2j5-mUzl-0XW/s1600/Screen+Shot+2016-02-13+at+14.25.07.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgtj8W9jLssimyTj5NEtIaXmEz_6973tb2FeOh0aoo2UuxBuHyvqHiDIEwowoJzsLxWCpZuLTmDJNP4-oGh605pvXc_POzAwqJLprgy5JNIt_Y_5RV0zNYsuRjoZBwjwkVh2j5-mUzl-0XW/s320/Screen+Shot+2016-02-13+at+14.25.07.png" width="275" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDFgVupmq8w_VbsN_62oZ0jYcfmoyfkTHKAMeQqr3Pr4KdJGUriQGgK5sDFmYIuv6jjiU5X8zC2oPINWhVa3cIsUt92G9T_43VJdiPFkKfM4oFLgpQb8ViR-DxHjO-0bbLOSJElwEwxK3i/s1600/Screen+Shot+2016-02-13+at+14.25.14.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDFgVupmq8w_VbsN_62oZ0jYcfmoyfkTHKAMeQqr3Pr4KdJGUriQGgK5sDFmYIuv6jjiU5X8zC2oPINWhVa3cIsUt92G9T_43VJdiPFkKfM4oFLgpQb8ViR-DxHjO-0bbLOSJElwEwxK3i/s320/Screen+Shot+2016-02-13+at+14.25.14.png" width="274" /></a><br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9URUcUZIuuNHKqpNHkTgrabNg_8o_V2lBn2AZTe0s5WukRrFc4sNcKOaYFI4vMKyq0BGo7MWWVOSOCPW9vIuuYKpB6SEMLfQB7dYEW3GJpzkg2iQSH8yhdSAUbNdmGqlOzwfNa3z7PHQ6/s1600/Screen+Shot+2016-02-13+at+14.25.20.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9URUcUZIuuNHKqpNHkTgrabNg_8o_V2lBn2AZTe0s5WukRrFc4sNcKOaYFI4vMKyq0BGo7MWWVOSOCPW9vIuuYKpB6SEMLfQB7dYEW3GJpzkg2iQSH8yhdSAUbNdmGqlOzwfNa3z7PHQ6/s320/Screen+Shot+2016-02-13+at+14.25.20.png" width="274" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQt1ljuyk1XFxxgId1ORduDu50voXlkgHCCFb_gKOYyQOvba5Bk7BJpzwgeT10wwbQbjfwPxqwz_gFK_5FZJ9mM_bvG3oehrohlPO2qTN1lCAZXzhSU6heI7NtJbfvRNCKQaXggdGCGis8/s1600/Screen+Shot+2016-02-13+at+14.25.28.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQt1ljuyk1XFxxgId1ORduDu50voXlkgHCCFb_gKOYyQOvba5Bk7BJpzwgeT10wwbQbjfwPxqwz_gFK_5FZJ9mM_bvG3oehrohlPO2qTN1lCAZXzhSU6heI7NtJbfvRNCKQaXggdGCGis8/s320/Screen+Shot+2016-02-13+at+14.25.28.png" width="280" /></a><br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJ6jmzwUITELJ3NVbJ3MINNVaEQQ99oHZSo0nZ_irKXevZabSu1QXcpBbZGSElnasETRW_cyOYuxDISECz4Q2S_wEv1HYzfZEJHeso3xw5ahq2mebjGYUM6CwPRyRgdFvk-DcHvkfC8AZU/s1600/Screen+Shot+2016-02-13+at+14.25.36.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJ6jmzwUITELJ3NVbJ3MINNVaEQQ99oHZSo0nZ_irKXevZabSu1QXcpBbZGSElnasETRW_cyOYuxDISECz4Q2S_wEv1HYzfZEJHeso3xw5ahq2mebjGYUM6CwPRyRgdFvk-DcHvkfC8AZU/s320/Screen+Shot+2016-02-13+at+14.25.36.png" width="277" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhmuBqdDblBkxRd81IRxgoq3Ojv2zJqLsdmgH3rj7WNfbeqJp_9LZlneWRbFuc3tTq8Z57sHpgrlw8C1kixKNNZLYmE3JuczR53iJA9yK28w1ZcTtaaCxI8w7wOqA8Zj72_AcYFIHWRKoco/s1600/Screen+Shot+2016-02-13+at+14.25.45.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhmuBqdDblBkxRd81IRxgoq3Ojv2zJqLsdmgH3rj7WNfbeqJp_9LZlneWRbFuc3tTq8Z57sHpgrlw8C1kixKNNZLYmE3JuczR53iJA9yK28w1ZcTtaaCxI8w7wOqA8Zj72_AcYFIHWRKoco/s320/Screen+Shot+2016-02-13+at+14.25.45.png" width="279" /></a><br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjpHAN04-H7EJo2tRfstmKWFaZP63D5_fnLNJbv3ODO-M4XsPF7nqLgp0c-mYLzrEGHyb6VbP5NeWenARMU_XKuxChSIfjabJ36OG7Vgl6LdU1j9hGITePbl3MT1-KQo8G0m8dwqlz8rreL/s1600/Screen+Shot+2016-02-13+at+14.25.56.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjpHAN04-H7EJo2tRfstmKWFaZP63D5_fnLNJbv3ODO-M4XsPF7nqLgp0c-mYLzrEGHyb6VbP5NeWenARMU_XKuxChSIfjabJ36OG7Vgl6LdU1j9hGITePbl3MT1-KQo8G0m8dwqlz8rreL/s320/Screen+Shot+2016-02-13+at+14.25.56.png" width="277" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8xaafUM5vOhSBNtu2D5iDqFrnos_9tpj0isM9kMtEBQXHFxiyI4tk0dpa2Roi7DElfmpNHmdMaaBIxFD1pcwC9wBvhct1n5LIO-UcXgpxljcmpHBUprbWx8alhkxz_hraGDV-iXjuUyo4/s1600/Screen+Shot+2016-02-13+at+14.26.04.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8xaafUM5vOhSBNtu2D5iDqFrnos_9tpj0isM9kMtEBQXHFxiyI4tk0dpa2Roi7DElfmpNHmdMaaBIxFD1pcwC9wBvhct1n5LIO-UcXgpxljcmpHBUprbWx8alhkxz_hraGDV-iXjuUyo4/s320/Screen+Shot+2016-02-13+at+14.26.04.png" width="276" /></a><br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhK1nGXw1U4C5Sa2VHPS1sP4GLE6nD9BRcnQifJ9wDDYRFdHNm_nfRLeAgqHRsp8oKqGaxZY-3nQQf8HMF864vELjRDd6Mx2S5fG54DN2TFyE70O88aG6loKd91lfy0Q7UXePMd3NJ25Z6U/s1600/Screen+Shot+2016-02-13+at+14.26.14.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhK1nGXw1U4C5Sa2VHPS1sP4GLE6nD9BRcnQifJ9wDDYRFdHNm_nfRLeAgqHRsp8oKqGaxZY-3nQQf8HMF864vELjRDd6Mx2S5fG54DN2TFyE70O88aG6loKd91lfy0Q7UXePMd3NJ25Z6U/s320/Screen+Shot+2016-02-13+at+14.26.14.png" width="277" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkTVEfvvyumdHCyiCD8ygK7rn1aZtOly44_PPqQMWnUtk6oxs5p6AXcgYk0qHChchimEfunJpWPRarq9yavTJp0thhm9y_aMIzYyM0wL4yrWbsw041IGmCtWXOyzow0JHlnsMtWWizjyKS/s1600/Screen+Shot+2016-02-13+at+14.26.24.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkTVEfvvyumdHCyiCD8ygK7rn1aZtOly44_PPqQMWnUtk6oxs5p6AXcgYk0qHChchimEfunJpWPRarq9yavTJp0thhm9y_aMIzYyM0wL4yrWbsw041IGmCtWXOyzow0JHlnsMtWWizjyKS/s320/Screen+Shot+2016-02-13+at+14.26.24.png" width="274" /></a><br />
<br />Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com0tag:blogger.com,1999:blog-7760425276056032223.post-78911832081979530352016-01-31T16:55:00.001-08:002017-08-15T10:16:05.090-07:00Installing mysqlworkbench and mysql utils in MacOSX Yosemite<br />
<br />
<br />
<br />
<h2>
Installing MySQL, mysqlworkbench and mysql-utils in Yosemite</h2>
<div>
<br /></div>
<div>
For installing things like *nix in MacOSX I use homebrew.</div>
<div>
<br /></div>
<div>
Homebrew is fantastic for installing tar.gz files in textual way, but there are some packages that are only in .dmg format and you need to use a package manager that knows how to install this GUI based installers. This package manager is <a href="http://caskroom.io/" target="_blank">homebrew cask </a>(caskroom project)</div>
<div>
<br /></div>
<div>
First you need to install mysql</div>
<div>
<br /></div>
<div>
<div class="shell">
<pre>$ brew install mysql
$ mysql.server start
# and optionaly make it more secure
$ mysql_secure_installation
</pre>
<pre></pre>
</div>
</div>
<div>
<br /></div>
<div>
Then you need to 'tap' the cask project and then get your 'cask' before 'pouring' it.<br />
<br />
<br />
<div class="shell">
<pre>$ brew tap caskroom/cask
# optional
$ brew install homebrew/completions/brew-cask-completion
## install workbench and utils
$ brew cask install mysqlworkbench
# mysqlworkbench staged at '/opt/homebrew-cask/Caskroom/mysqlworkbench/6.3.6'
# Symlinking App 'MySQLWorkbench.app' to '/Users/pmg/Applications/MySQLWorkbench.app'
$ brew cask install mysql-utilities
# mysql-utilities staged at '/opt/homebrew-cask/Caskroom/mysql-utilities/1.5.6'
</pre>
</div>
<br />
Note that workbench is symlinked to your home apps not the main Application folder<br />
<br />
<br />
<div class="p1">
<span class="box">~/Applications/MySQLWorkbench.app</span><br />
<br />
Launch it and you will get this window with the MySQL Utils icon at the top right
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQGrGPcLRFEmS6v-dAebl-5Kh7RiTrnI62wunH57Ii4gMqBsMOOAsCnG2n1heHfKVmWotDpxP9KKNum1d4c-fueneQW9Tt7St_uMcEjndcvPJ2Tq8VR6QQ48rpGevOCbaPPwuYtA11bCBD/s1600/Screen+Shot+2016-02-01+at+00.58.21.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="393" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQGrGPcLRFEmS6v-dAebl-5Kh7RiTrnI62wunH57Ii4gMqBsMOOAsCnG2n1heHfKVmWotDpxP9KKNum1d4c-fueneQW9Tt7St_uMcEjndcvPJ2Tq8VR6QQ48rpGevOCbaPPwuYtA11bCBD/s640/Screen+Shot+2016-02-01+at+00.58.21.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
When you click in the MySQL Utilities.... OPS!! it does not work!<br />
<br />
MySQL Utils launch a console and the error message is<br />
<br />
<div class="shell">
<pre>$ mysqluc -e "help utilities"
ERROR: The MySQL Connector/Python module was not found. MySQL Utilities requires the connector to be installed. Please check your paths or download and install the Connector/Python from http://dev.mysql.com.</pre>
</div>
<div class="p1">
<span class="s1"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
OK, let's find the connector googleing it. And find it<br />
<br />
<br />
<div class="box">
[EDIT] I have just missed that the mysql-connector-python is also in cask and you can install with `brew cask install mysql-connector-python`<u> BUT IT DID NOT WORK FOR ME</u><br />
<br />
<div class="p1">
<span class="s1"><b>$ brew search mysq</b>l</span></div>
<div class="p1">
<span class="s1">automysqlbackup </span><span class="s2">mysql </span><span class="s3">✔</span><span class="s1"> mysql++ mysql-cluster mysql-connector-c mysql-connector-c++ mysql-sandbox mysql-search-replace mysqltuner </span></div>
<div class="p1">
<span class="s1">homebrew/php/php53-mysqlnd_ms homebrew/php/php55-mysqlnd_ms homebrew/versions/mysql51 homebrew/versions/mysql56 Caskroom/cask/mysql-utilities Caskroom/cask/navicat-for-mysql </span></div>
<div class="p1">
<span class="s1">homebrew/php/php54-mysqlnd_ms homebrew/php/php56-mysqlnd_ms homebrew/versions/mysql55 <span style="background-color: yellow;"><b>Caskroom/cask/mysql-connector-python</b> </span> Caskroom/cask/mysqlworkbench </span></div>
<br />
<div class="p2">
<br /></div>
<div class="p2">
<br /></div>
<div class="p1">
<span class="s1"><b>$ brew cask install mysql-connector-python</b></span></div>
<div class="p1">
<b><span style="color: red;"><span class="s2">Error</span><span class="s1">: No available Cask for mysql-connector-python</span></span></b></div>
<div class="p2">
</div>
<div class="p1">
<b><span style="color: red;"><span class="s2">Error</span><span class="s1">: nothing to install</span></span></b></div>
<div class="p2">
<span class="s4"><br /></span></div>
</div>
<br />
at <b><a href="http://dev.mysql.com/downloads/connector/python/">http://dev.mysql.com/downloads/connector/python/</a> </b>but instead of selecting Mac choose "platform independent"<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQ7k0SrO0y8Lt4AYqg5VhyqHX9cln37k7vbLY7Wp-ji4pb2s7NTpxqLkTKkKNcUjhw9lLHi7UHZ5xXxAH04DIc5zzU7yXhwIfowd9YH158NoZwfvTzMkcTywfmGZBeMCjFsr3xVX1fAJfF/s1600/Screen+Shot+2016-02-01+at+01.35.17.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="308" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQ7k0SrO0y8Lt4AYqg5VhyqHX9cln37k7vbLY7Wp-ji4pb2s7NTpxqLkTKkKNcUjhw9lLHi7UHZ5xXxAH04DIc5zzU7yXhwIfowd9YH158NoZwfvTzMkcTywfmGZBeMCjFsr3xVX1fAJfF/s640/Screen+Shot+2016-02-01+at+01.35.17.png" width="640" /></a></div>
<br />
choose download and and then copy the link from "no thanks, just start download"<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEja1u3Y58rRksXmVIbQsSXTcK02c9Ci7IxeJbj7bDCKrTDG9C5yIt3DQEzyfx3f-EM6qndKuBdYar0FBXjjBwuk9-n7bcZ3kQhRis0S9tcHOgVF0qh0gYa8hnW5g3qdhSKfylnt9Q6UhHM3/s1600/Screen+Shot+2016-02-01+at+01.37.07.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="290" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEja1u3Y58rRksXmVIbQsSXTcK02c9Ci7IxeJbj7bDCKrTDG9C5yIt3DQEzyfx3f-EM6qndKuBdYar0FBXjjBwuk9-n7bcZ3kQhRis0S9tcHOgVF0qh0gYa8hnW5g3qdhSKfylnt9Q6UhHM3/s640/Screen+Shot+2016-02-01+at+01.37.07.png" width="640" /></a></div>
<br />
<br />
Go back to your shell and download it (you can do it in the mysql-utils dir just to have all together)<br />
<br />
<div class="shell">
<pre>$ cd /opt/homebrew-cask/Caskroom/mysql-utilities/
$ wget http://dev.mysql.com/get/Downloads/Connector-Python/mysql-connector-python-2.1.3.tar.gz
$ tar -zxvf mysql-connector-python-2.1.3.tar.gz
$ cd mysql-connector-python-2.1.3/
# install for both python
$ sudo python ./setup.py install
$ sudo python3 ./setup.py install
</ pre>
</pre>
</div>
<br />
Now when you click in the MySQL-Utils you have a working connection and you see this in the launched terminal<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZ6qMYNUpS5fiktbixbMxk25g_dQN-l95xc6rkT92i-JgRCMr1CNHZCgwnR1fnByIlxuHtrvZURozYm4kVe43306Rb9uwnBjfV1ccoNkflZqZHMo8EN-L6ApZlvR1RfpcU6ezRMn-prXWr/s1600/Screen+Shot+2016-02-01+at+00.58.39.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="270" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZ6qMYNUpS5fiktbixbMxk25g_dQN-l95xc6rkT92i-JgRCMr1CNHZCgwnR1fnByIlxuHtrvZURozYm4kVe43306Rb9uwnBjfV1ccoNkflZqZHMo8EN-L6ApZlvR1RfpcU6ezRMn-prXWr/s320/Screen+Shot+2016-02-01+at+00.58.39.png" width="320" /></a></div>
<br />
<br />
<b>P.S.</b><br />
<b><br /></b>
I have tried to install the connector from python using pip but it did not work due a problem with ConfigParser. I did not investigate it further because I have already a working solution.<br />
<br />
<div class="code">
<br />
<div class="p1">
<span class="s1"> MacBook-Pro:[~]</span></div>
<div class="p1">
<span class="s1"> $ pip3 install MySQL-python</span></div>
<div class="p1">
<span class="s1"> Collecting MySQL-python</span></div>
<div class="p1">
<span class="s1"> Downloading MySQL-python-1.2.5.zip (108kB)</span></div>
<div class="p1">
<span class="s1"> Complete output from command python setup.py egg_info:</span></div>
<div class="p1">
<span class="s1"> Traceback (most recent call last):</span></div>
<div class="p1">
<span class="s1"> File "<string>", line 20, in <module></span></div>
<div class="p1">
<span class="s1"> File "/private/var/folders/m0/spbgfzvd17jdfqgx90w1_hx40000gn/T/pip-build-0nif_qmi/MySQL-python/setup.py", line 13, in <module></span></div>
<div class="p1">
<span class="s1"> from setup_posix import get_config</span></div>
<div class="p1">
<span class="s1"> File "/private/var/folders/m0/spbgfzvd17jdfqgx90w1_hx40000gn/T/pip-build-0nif_qmi/MySQL-python/setup_posix.py", line 2, in <module></span></div>
<div class="p1">
<span class="s1"> from ConfigParser import SafeConfigParser</span></div>
<div class="p1">
<span class="s1"> ImportError: <b>No module named 'ConfigParser'</b></span></div>
<div class="p3">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1"> ----------------------------------------</span></div>
<div class="p1">
<span class="s1"> Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/m0/spbgfzvd17jdfqgx90w1_hx40000gn/T/pip-build-0nif_qmi/MySQL-python</span></div>
<div class="p1">
<span class="s1"> You are using pip version 7.1.2, however version 8.0.2 is available.</span></div>
<div class="p1">
<span class="s1"> You should consider upgrading via the 'pip install --upgrade pip' command.</span></div>
<div class="p1">
<span class="s1"> </span></div>
<div class="p1">
<span class="s1"> MacBook-Pro:[~]</span></div>
<div class="p1">
<span class="s1"> <b>$ pip3 install ConfigParser</b></span></div>
<div class="p1">
<span class="s1"> Collecting ConfigParser</span></div>
<div class="p1">
<span class="s1"> Downloading configparser-3.3.0r2.tar.gz</span></div>
<div class="p1">
<span class="s1"> Complete output from command python setup.py egg_info:</span></div>
<div class="p1">
<span class="s1"> Traceback (most recent call last):</span></div>
<div class="p1">
<span class="s1"> File "<string>", line 3, in <module></span></div>
<div class="p1">
<span class="s1"> File "/usr/local/lib/python3.5/site-packages/setuptools/__init__.py", line 5, in <module></span></div>
<div class="p1">
<span class="s1"> import distutils.core</span></div>
<div class="p1">
<span class="s1"> File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/core.py", line 18, in <module></span></div>
<div class="p1">
<span class="s1"> from distutils.config import PyPIRCCommand</span></div>
<div class="p1">
<span class="s1"> File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/config.py", line 7, in <module></span></div>
<div class="p1">
<span class="s1"> from configparser import ConfigParser</span></div>
<div class="p1">
<span class="s1"> File "/private/var/folders/m0/spbgfzvd17jdfqgx90w1_hx40000gn/T/pip-build-0ouo8j8z/ConfigParser/configparser.py", line 397</span></div>
<div class="p1">
<span class="s1"> _KEYCRE = re.compile(ur"%\(([^)]+)\)s")</span></div>
<div class="p1">
<span class="s1"> ^</span></div>
<div class="p1">
<span class="s1"> <b>SyntaxError: invalid syntax</b></span></div>
<div class="p3">
<span class="s1"></span><br /></div>
<br />
<div class="p1">
<span class="s1"> ----------------------------------------</span></div>
</div>
</div>
</div>
Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com2tag:blogger.com,1999:blog-7760425276056032223.post-60701991748156598332014-10-15T13:38:00.002-07:002014-10-15T13:41:50.662-07:00new security vulnerability called POODLEThere Is a New Security Vulnerability Named POODLE, and It Is Not Cute: <a href="http://www.wired.com/2014/10/poodle-explained/">http://www.wired.com/2014/10/poodle-explained/</a><br />
<br />
<a href="https://supportforums.cisco.com/discussion/12326341/sslv3-poodle-vulnerability">https://supportforums.cisco.com/discussion/12326341/sslv3-poodle-vulnerability</a><br />
<br />
<a href="http://www.zdnet.com/google-reveals-major-flaw-in-outdated-but-widely-used-ssl-protocol-7000034677/">http://www.zdnet.com/google-reveals-major-flaw-in-outdated-but-widely-used-ssl-protocol-7000034677/</a><br />
<br />
<br />
<div class="p1">
POODLE affects SSLv3 or version 3 of the Secure Sockets Layer protocol, which is used to encrypt traffic between a browser and a web site or between a user’s email client and mail server. It’s not as serious as the recent Heartbleed and Shellshock vulnerabilities, <span class="s1">but POODLE could allow an attacker to hijack and decrypt the session cookie that identifies you to a service like Twitter or Google, and then take over your accounts without needing your password.</span></div>
<div class="p1">
<span class="s1"><br /></span></div>
<div class="p1">
<span class="s1">According to the team's Bodo Möller: "This vulnerability allows the plaintext of secure connections to be calculated by a network attacker."</span></div>
<br />
This is a client security hole. This is a vulnerability in the old SSLv3 (relaying in 80's code), but even if you have all set for using TLS, in many applications the server can trick you to downgrade to SSL and then it can stole you some security cookies.<br />
<br />
This vulnerability is a risk in public wifi but not at home or work.<br />
<br />
<br />
<div class="p1">
<br /></div>
<div class="p1">
------------------</div>
<br />
Mozilla says that it is making Firefox 34 safe from POODLE by disabling SSL 3.0 by default. The code which does this is already baked into the Nightly channel, and will make its way to the Aurora and Beta channels also "in the next few weeks".<br />
<br />
----------<br />
MicroSoft Advisory and Workarounds:<br />
<br />
https://technet.microsoft.com/en-us/library/security/3009008.aspx<br />
This workaround involves Group Policy Editor. For those using versions of Windows that don't include GPE simply go to Control Panel/Internet Options/Advanced/Security and uncheck the box for SSL 3.0 (SSL 2.0 should already be unchecked).Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com0tag:blogger.com,1999:blog-7760425276056032223.post-60958581075166869662014-08-07T02:24:00.001-07:002014-08-07T02:24:53.868-07:00Ubuntu 14.04 new Ibus is interfering with default emacs keybinding control-space
<p>In emacs you select text marking a starting selection point with control-space. This is a fundamental key binding in emacs. Now in the new version of Ubuntu (14.04) the default service for controlling multilingual input in linux has been changed to <a href="http://en.wikipedia.org/wiki/Intelligent_Input_Bus">Ibus</a>. The issue is that Ibus default key binding to control-space. </p>
<p>If you want to use emacs you probably should change this to something else. Go to a terminal and write 'ibus-setup' and at the right of the input method line you will find three dots '...' click there and change your key binding to Ibus pressing in the three dots in the key code line and then clik in disable. The text will change to 'new accelerator', press your combination of keys and you are done. I have changed mine to super-alt-space (super is the windows key).</p>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixmspfjDNvhns1Zi2Dp5WcHDixYRXfyR5WRVKI_z_lb9uvQ-NXr9FiBjbNGVyDFYhnm7yRMA0vpfNLw5_YHYy2uMK7uYguHuXG7k4KnpySVsUdZq-Clw6bhFa0M-ZQkCjP1qQKJ0pIuy20/s1600/ibus_conf.png" imageanchor="1" ><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixmspfjDNvhns1Zi2Dp5WcHDixYRXfyR5WRVKI_z_lb9uvQ-NXr9FiBjbNGVyDFYhnm7yRMA0vpfNLw5_YHYy2uMK7uYguHuXG7k4KnpySVsUdZq-Clw6bhFa0M-ZQkCjP1qQKJ0pIuy20/s320/ibus_conf.png" /></a>Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com0tag:blogger.com,1999:blog-7760425276056032223.post-82184756104462586302014-02-27T19:09:00.001-08:002014-02-28T15:44:50.044-08:00perl gzip libraries (probably zlib issue) does not play well with bgzip files.<br />
In bioinformatcis, <a href="http://manpages.ubuntu.com/manpages/raring/man1/bgzip.1.html">bgzip</a> files are important for random access to big files . Bgzip is a program modified from gzip program that uses block compression and is fully backwards compatible with gzip.
But I have issues when using bgzip compressed vcf files with Perl scripts that uses IO::Uncompress::Gunzip (that I believe it uses zlib under the hood). A similar problem happen to my recently with snpeff program (Java). In both cases the data is decompressed but truncated after a few hundred lines aprox.
I could be totally wrong but I was wondering if zlib (or whatever gzip compatible library they are using) is getting confused with the bgzip bloks and only processing one or a few of them leaving the output incomplete.
perl code that does not work:<br />
<br />
<div class="code">
<pre>#!/usr/bin/env perl
use strict;
use IO::Uncompress::Gunzip qw(gunzip $GunzipError) ;
my $infile = shift;
my $infh = IO::Uncompress::Gunzip->new( $infile )
or die "IO::Uncompress::Gunzip failed: $GunzipError\n";
my $line_count = 0;
while (my $line=<$infh>){
$line_count++
}
print "total lines read = $line_count\n";
</pre>
</div>
This gives 419 lines
<br />
<div class="code">
<pre> $ perl /home/pmg/tmp/test_zlib-bgzip.pl varsit.vcf.gz
total lines read = 419
</pre>
</div>
but using open with gzip pipe works:
<br />
<div class="code">
<pre> #!/usr/bin/env perl
use strict;
# I can use bgzip intead gzip
my $infile = shift;
open(my $infh , 'gzip -dc '.$infile.' |');
my $line_count = 0;
while (my $line=<$infh>){
$line_count++
}
print "total lines read = $line_count\n";
</pre>
</div>
Gives the expected number of lines
<br />
<div class="code">
<pre> $ perl /home/pmg/tmp/test_gzip-bgzip.pl varsit.vcf.gz
total lines read = 652829
</pre>
</div>
I googled about and I was unable to find quickly any relevant entry, but this is something that I am sure other people would have already faced. Do someone have a clue about why is this happening? I am using ubuntu 12.04.4 with perl 5.16<br />
<br />
[UPDATE 2014-02-28]: finally a clue come from <a href="http://www.biostars.org/p/94240/#94280">biostars </a>where Heng Li remind me a footnote in the SAM specs about a java library for gzip that only sees first block of bgzip when decompressing. Seems that Perl gzip implementations had the same problem.Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com0tag:blogger.com,1999:blog-7760425276056032223.post-48689772328960287292014-02-24T08:53:00.002-08:002014-02-24T08:55:58.423-08:00automatic encrypting decrypting with emacsI have been for long time wondering if there was a way to automatically encrypt and decrypt files when accessing them, not leaving an unencrypted file around.<br />
<br />
Today I have an epiphany: "let's see if emacs can do that". As an emacs power user I felt as an idiot. Emacs has in the core this ability since emacs 23!! and I never realized about it :-(.<br />
<br />
Emacs has incorporated the EasyPG code into its core.<br />
<a href="http://www.emacswiki.org/emacs/EasyPG">http://www.emacswiki.org/emacs/EasyPG</a><br />
<br />
Things can not be more easy. Put in your .emacs<br />
<br />
<div class="code">
<pre>(require 'epa-file)
(epa-file-enable)
;; as it is annoying to be asked if I want passphrase or publick/private key
;; I set passphrase as default
(setq epa-file-select-keys nil)
</pre>
</div>
I also would like to have a gpg agent cache. In ubuntu 12.04 the package is called <b>gnupg-agent</b><br />
<br />
<div class="code">
<pre>sudo apt-get install gnupg-agent
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
libassuan0 pinentry-gtk2
Suggested packages:
pinentry-doc
The following NEW packages will be installed:
gnupg-agent libassuan0 pinentry-gtk2
0 upgraded, 3 newly installed, 0 to remove and 1 not upgraded. </pre>
</div>
<br />
<br />
Now you only need to open a file ended with .gpg and automatically emacs would ask you for the pasword for decryption and again for saving. No unencrypted temporary files are stored, or at least I am not aware of them.Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com0tag:blogger.com,1999:blog-7760425276056032223.post-64618730127229662232013-08-20T05:56:00.003-07:002013-08-20T06:00:25.203-07:00Reducing batch effects in methylation analysisIn our sequencing facility we have processed 4 illumina 450K methylation chips for the same study at 2 diferent times. We took care of spreading case and controls among all the chips. We can see a clear time-of-processing batch effect:<br />
<br />
The chips were run in two different days one month apart. first 24 samples and then the other 24 samples.<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4jHWw1Chhr3qCZiEBlNTCYV7Vzn48WHskcos0i0B-qy7lXPcBYtBW_-uDzzAFfgg69AtU9Rh5G0zviihcj3JxY9KJ2XYvpiuwM-qXWUg8tND8UPew2nWce-59ZIxpXLcml0LFM9Rpo-kl/s1600/qc-negative-controls.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="441" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4jHWw1Chhr3qCZiEBlNTCYV7Vzn48WHskcos0i0B-qy7lXPcBYtBW_-uDzzAFfgg69AtU9Rh5G0zviihcj3JxY9KJ2XYvpiuwM-qXWUg8tND8UPew2nWce-59ZIxpXLcml0LFM9Rpo-kl/s640/qc-negative-controls.png" width="640" /></a><br />
<br />
The negative controls of the second batch have a wider range.<br />
<br />
And wen looking at the MDS and PCA we can see that the main parameter for grouping samples is the run day (the two chips starting with 835... where done the same day and the other two another day) <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEggNNw42hsvqKK_hYDFb1qtTmdviIaj0N0-rVgwM-wv5uvONX7URhx6294EqcW81ZMOVtBFJ2NGBTWzmKvpMQp2buVYPq3dZxrfnw-Yqk7klOkhDEZhoh9VjhQ9EAtSYLs7SoIdKZF0_fDM/s1600/batch-effect-PCA.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="292" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEggNNw42hsvqKK_hYDFb1qtTmdviIaj0N0-rVgwM-wv5uvONX7URhx6294EqcW81ZMOVtBFJ2NGBTWzmKvpMQp2buVYPq3dZxrfnw-Yqk7klOkhDEZhoh9VjhQ9EAtSYLs7SoIdKZF0_fDM/s320/batch-effect-PCA.png" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjtA3EIhdEHfYZPNebKbkgcMAMLlxXyoV3LgsqoETokYYqunkn19Q-TbNijPdCImZB9FtW9QD3R_4lOUvBl14hs8qUUYPtkY0OwroxDlRDy_kXmd7yxr-zE9krvucZ64MXyofL8uB4Z7GlX/s1600/batch-effect-MDS.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="294" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjtA3EIhdEHfYZPNebKbkgcMAMLlxXyoV3LgsqoETokYYqunkn19Q-TbNijPdCImZB9FtW9QD3R_4lOUvBl14hs8qUUYPtkY0OwroxDlRDy_kXmd7yxr-zE9krvucZ64MXyofL8uB4Z7GlX/s320/batch-effect-MDS.png" width="320" /></a></div>
<br />
I have used the nice <a href="http://rnbeads.mpi-inf.mpg.de/">rnbeads</a> R package for reporting the batch effects. <br />
<br />
Now I am exploring two methods of correcting the batch effect:<br />
<ol>
<li> limma using the batch date as block effect</li>
<ul>
<li>design <- model.matrix(~Block+Treatment)<br /> </li>
</ul>
<li>ComBat in the R SVA package for correcting this </li>
</ol>
I will post the results soon. Meanwhile any comment for dealing with batch effects in 450K chips is more than wellcome. <br />
Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com0tag:blogger.com,1999:blog-7760425276056032223.post-87834535197537571192013-07-02T15:07:00.001-07:002013-07-02T15:07:27.293-07:00where is vnc viewer vinagre storing its history of connections?not easy to find manually so I did<br />
<br />
<div class="code">
find ~/ -name vinagre<br />
</div>
<br />
.local/share/vinagre<br />
<br />
<div class="box">
analysis[~] cat .local/share/vinagre/history<br />
localhost:8900<br />localhost:8910<br />localhost:8920<br />localhost:8930<br />localhost:8907<br />
</div>Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com0tag:blogger.com,1999:blog-7760425276056032223.post-33109106685983750272013-07-02T14:11:00.001-07:002013-07-02T14:12:19.267-07:00unbuntu 12.10 install hangs in first boot 'waiting for battery status'. Seems it was a problem with my graphics card.I have two new i7 servers and I installed ubuntu 12.10. Unfortunately after installation (without network) both of them did not started the Xs and stoped with a message ''waiting for battery status". After googling for answers for this issue, I saw that many of the solutions either fixed it feedling with the display manager or Xauthorities or Xorgs or simply being practical and starting a Xserver from ctr-alt-f1 but for some users none of these actions fixed the problem.<br />
<br />
Seems that all solutions are related with the Xs or display. In my case both computers have an Asus GForce 210 graphical card but I have a third identical server without the card has no problem. So my first guess was a possible issue with the linux drivers for the card. So first solution was to go to the BIOS and change the GPU to iGPU (the integrated one). Voilà, the machine boots into graphical mode with no problem.<br />
<br />
Once I obtained the IP for my mac address, I upgraded (on computer only upgrade, and the odter dist-upgrade to 13.04) and now I am able to boot correctly using my PCI GPU card from both computers. <br />
<br />
So if you find this problem (that seems not so rare, but I have never seen before and I have intalled a lot of ubuntus in a lot of machines since the 'noname-distribution'), before doing other things try to upgrade your computer to the same or a newer distro. If none of the web suggestions fix the problem you can always try to use your integrated GPU o change your graphics card.Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com0tag:blogger.com,1999:blog-7760425276056032223.post-36631017921996131072013-06-24T21:31:00.000-07:002013-06-24T21:31:38.510-07:00fixing "Error while parsing file:freemind.main.XMLParseException: XML Parse Exception during parsing of a map element at line 1: Expected: ="I am a fan of mindmap programs. I am all the time using both, FreeMind and Xmind.<br />
<br />
Xmind is cute and has all the bells and whistles needed to enjoy the ride, but it has a resources hungry monster under the hood and is very slow when you have more than 6 or 7 big mindmaps with more than 6 or 7 tabs in each of them.<br />
<br />
For brainstorming and quick and simple mindmaps FreeMind is my piece of choice. I love its simplicity and the low level of resources usage it has compared with Xmind.<br />
<br />
Under this duality I thanks the import and export capabilities of XMind. It makes the life easier..... until java came along.<br />
<br />
Xmind exports to freemind 0.8 format. But now the stable version of FreeMind is 0.9. FreeMind 0.9 due to a java bug does not always opens FreeMind 0.8 maps, and fails with :<br />
<br />
<div class="box">
Error while parsing file:freemind.main.XMLParseException: XML Parse Exception during parsing of a map element at line 1: Expected: =
</div>
<br />
This has been also reported here:<br />
<a href="http://sourceforge.net/p/freemind/discussion/330666/thread/d09d930c/?limit=50">http://sourceforge.net/p/freemind/discussion/330666/thread/d09d930c/?limit=50</a><br />
<br />
And the solution is to install the latest FreeMind. In my case the 1.0 RC4<br />
<br />
<a href="http://sourceforge.net/projects/freemind/files/freemind-unstable/1.0.0_RC4/freemind-bin-max-1.0.0_RC_4.zip/download">http://sourceforge.net/projects/freemind/files/freemind-unstable/1.0.0_RC4/freemind-bin-max-1.0.0_RC_4.zip/download</a><br />
<br />Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com4tag:blogger.com,1999:blog-7760425276056032223.post-9786701141294482382013-05-13T10:47:00.000-07:002013-10-30T03:10:26.677-07:00Issues Installing R package ncdfJust for future reference:<br />
<br />
<b>Short story:</b><br />
before installing ncdf in R intall in your linux <b>libnetcdf-dev</b> and <b>netcdf-bin</b><br />
<b><br /></b>
<b>Long story:</b><br />
<br />
I found a problem installing the R package ncdf:<br />
<br />
<div class="code">
> install.package('ncdf')<br />
<br />
|checking whether we are using the GNU C compiler... yes<br />
|checking whether gcc -std=gnu99 accepts -g... yes<br />
|checking for gcc -std=gnu99 option to accept ISO C89... none needed<br />
|checking how to run the C preprocessor... gcc -std=gnu99 -E<br />
|checking for grep that handles long lines and -e... /bin/grep<br />
|checking for egrep... /bin/grep -E<br />
|checking for ANSI C header files... no<br />
|checking for sys/types.h... no<br />
|checking for sys/stat.h... no<br />
|checking for stdlib.h... no<br />
|checking for string.h... no<br />
|checking for memory.h... no<br />
|checking for strings.h... no<br />
|checking for inttypes.h... no<br />
|checking for stdint.h... no<br />
|checking for unistd.h... no<br />
|checking netcdf.h usability... no<br />
|checking netcdf.h presence... no<br />
|checking for netcdf.h... no<br />
|configure: error: netcdf header netcdf.h not found<br />
|ERROR: configuration failed for package ‘ncdf’<br />
|* removing ‘/usr/local/lib/R/site-library/ncdf’<br />
<div>
<br /></div>
</div>
<br />
Well I though, this is the classical "install the -devel package first" but alas, I have the netcdf.h in /usr/include !!! and I did already intalled the libnetcdf-dev package in my biolinux-7 (ubuntu 12.04)<br />
<br />
I went to the tmp folder where the R packages was downloaded and tried to install it by hand. I read the <b>INSTALL </b> file and tried adding environmental variables:<br />
<br />
<br />
<div class="box">
The location of the netcdf library can be specified by the environment<br />
variable NETCDF_LIB or the configure argument --with-netcdf-lib.</div>
<div>
<br /></div>
<div>
or adding FLAGS to ./configure</div>
<div>
<br /></div>
<div>
<div class="code">
LDFLAGS=-L/usr/lib CPPFLAGS=-I/usr/include sh configure</div>
</div>
<div>
<br /></div>
<div>
Nothing was working.</div>
<div>
<br /></div>
<div>
Finally I remembered that at the beginning of the INSTALL file it is mentioned that the program nc-config is a helper for the installer to find out the location of the dev files, but as it was not installed and it is said in this file that is optional I skipped this part during my first trials. After failed several times to find out how to pass the paths of the dev files I decided to give a try to the nc-config route. I searched for it and here <a href="http://cirrus.ucsd.edu/~pierce/ncdf/install_ncdf_v4.html">http://cirrus.ucsd.edu/~pierce/ncdf/install_ncdf_v4.html</a> I read:</div>
<div>
<br /></div>
<div class="box">
The only issue is getting the libraries right. All the libraries used by the netcdf library, version 4, must be visible to the R software. To get a list of what libraries must be visible, run "nc-config --libs" (note: a recent version of the netcdf library, version 4, must be installed correctly for this command to work).</div>
<br />
<br />
But I did not have nc-config<br />
<br />
<div class="code">
root@pmg-analysis:/tmp/RtmpiNeIMZ/downloaded_packages/ncdf# nc-config --libs
The program 'nc-config' is currently not installed. You can install it by typing:
apt-get install netcdf-bin
</div>
<br />
So I installed it
<br />
<div class="code">
root@pmg-analysis:/tmp/RtmpiNeIMZ/downloaded_packages/ncdf# apt-get install netcdf-bin
</div>
<br />
And then knowing that the R package ncdf uses this program for finding the headers and libs for netcdf, I run the install.package in R:
<br />
<div class="code">
<br />
>install.packages('ncdf')<br />
Installing package into ‘/home/pmg/R/x86_64-pc-linux-gnu-library/3.0’<br />
(as ‘lib’ is unspecified)<br />
trying URL 'http://cran.univ-paris1.fr/src/contrib/ncdf_1.6.6.tar.gz'<br />
Content type 'application/x-gzip' length 79403 bytes (77 Kb)<br />
opened URL<br />
==================================================<br />
downloaded 77 Kb<br />
<br />
* installing *source* package ‘ncdf’ ...<br />
** package ‘ncdf’ successfully unpacked and MD5 sums checked<br />
checking for nc-config... /usr/bin/nc-config<br />
configure: creating ./config.status<br />
config.status: creating src/Makevars<br />
** libs<br />
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -I/usr/include -fpic -O3 -pipe -g -c ncdf.c -o ncdf.o<br />
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -I/usr/include -fpic -O3 -pipe -g -c ncdf2.c -o ncdf2.o<br />
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -I/usr/include -fpic -O3 -pipe -g -c ncdf3.c -o ncdf3.o<br />
gcc -std=gnu99 -shared -o ncdf.so ncdf.o ncdf2.o ncdf3.o -L/usr/lib -lnetcdf -L/usr/lib/R/lib -lR<br />
installing to /home/pmg/R/x86_64-pc-linux-gnu-library/3.0/ncdf/libs<br />
** R<br />
** preparing package for lazy loading<br />
** help<br />
*** installing help indices<br />
** building package indices<br />
** testing if installed package can be loaded<br />
* DONE (ncdf)</div>
<br />
Job done!!Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com7tag:blogger.com,1999:blog-7760425276056032223.post-448851757670612182013-02-10T15:47:00.000-08:002013-02-10T15:47:19.677-08:00the perils of Excel: any one can do it <span style="background-color: white; color: #333333; font-family: Georgia, 'Bitstream Charter', serif; font-size: 14px; line-height: 23px;"><br /></span>
<span style="background-color: white; color: #333333; font-family: Georgia, 'Bitstream Charter', serif; font-size: 14px; line-height: 23px;"><br /></span>
<span style="background-color: white; color: #333333; font-family: Georgia, 'Bitstream Charter', serif; font-size: 14px; line-height: 23px;">A nice story to read about the perils of using excel when you don't know what you are doing. Fools rush in where angels fear to trade.</span><br />
<span style="background-color: white; color: #333333; font-family: Georgia, 'Bitstream Charter', serif; font-size: 14px; line-height: 23px;"><br /></span>
<span style="background-color: white; line-height: 22.99715805053711px;"><span style="color: #333333; font-family: Georgia, Bitstream Charter, serif; font-size: x-small;"><a href="http://baselinescenario.com/2013/02/09/the-importance-of-excel/">http://baselinescenario.com/2013/02/09/the-importance-of-excel/</a></span></span><br />
<blockquote>
[...]<span style="color: #333333; font-family: Georgia, 'Bitstream Charter', serif; font-size: 14px; line-height: 23px;"><span style="background-color: white;">The new model “operated through a series </span><span style="background-color: #fff2cc;">of Excel spreadsheets, which had to be completed manually, by a process of copying and pasting data from one spreadsheet to another</span><span style="background-color: white;">.” The internal Model Review Group identified this problem as well as a few others, but approved the model, while saying that it should be automated and another significant flaw should be fixed.** After the London Whale trade blew up, the Model Review Group discovered that the model had not been automated and found several other errors. Most spectacularly,</span></span><br />
<blockquote style="border: 0px; color: #333333; font-family: Georgia, 'Bitstream Charter', serif; font-size: 14px; line-height: 23px; margin: 0px 30px 1.7em; outline: 0px; padding: 0px; quotes: ''; vertical-align: baseline;">
<div style="border: 0px; font-family: inherit; margin-bottom: 1.7em; outline: 0px; padding: 0px; vertical-align: baseline;">
<span style="background-color: white; font-style: inherit;">“</span><span style="background-color: #fff2cc;">After subtracting the old rate from the new rate, the spreadsheet divided by their sum instead of their average, as the modeler had intended</span><span style="background-color: white; font-style: inherit;">. This error likely had the effect of muting volatility by a factor of two and of lowering the VaR . . .”</span></div>
</blockquote>
[...]<br />
<span style="color: #333333; font-family: Georgia, 'Bitstream Charter', serif; font-size: 14px; line-height: 23px;"><span style="background-color: #fff2cc;">But while Excel the program is reasonably robust, the spreadsheets that people create with Excel are incredibly fragile</span><span style="background-color: white;">. </span><b style="background-color: #cfe2f3;">There is no way to trace where your data come from, there’s no audit trail (so you can overtype numbers and not know it), and there’s no easy way to test spreadsheets, for starters.</b><span style="background-color: white;"> The biggest problem is that anyone can create Excel spreadsheets—badly.</span><b style="background-color: white;"> </b><span style="background-color: #fff2cc;">Because it’s so easy to use, the creation of even important spreadsheets is not restricted to people who understand programming and do it in a methodical, well-documented way.</span><span style="background-color: white;">***</span></span><br />
[...]</blockquote>
Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com0tag:blogger.com,1999:blog-7760425276056032223.post-81719006739430349612013-02-10T04:40:00.000-08:002013-02-10T15:47:41.220-08:00The importance of logarithmic transformation in 'natural' data<br />
Reading the Edward Tuft book about data analysis in politics and policy<br />
<br />
<a href="http://www.edwardtufte.com/tufte/dapp/">http://www.edwardtufte.com/tufte/dapp/</a><br />
<br />
Edward Tuft is one of the gurus of Data Analysis visualization [0], and in this chapter [1] he show in a very didactic and clear way the importance of logarithmic transformation for data of naturally occurring counts.<br />
<span style="font-size: x-small;"> [0] <a href="http://en.wikipedia.org/wiki/Edward_Tufte">http://en.wikipedia.org/wiki/Edward_Tufte</a></span><br />
<span style="font-size: x-small;"> [1] <a href="http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0003uF">http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0003uF</a></span><br />
<br />
<h2>
The importance of logarithmic transformation</h2>
<div>
<br /></div>
<a href="http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0003uF" style="font-size: small;">http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0003uF</a><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlwI3w_Q7ZVzH1dRkKAjDJuTO9Bk8w5R-JYTPrUztCKmFRaW8ET1r9LI84kNLvJaUTj7EcIzmgskaboLYqeQ1tteeAd75wNMykd9rT0CmG3w4y7waXhtOUKRyyqzZTbNr3FkmFJYqR1KeS/s1600/+datanal+log_transformation_and_fitting_models-1+Tuft+2013-02-10.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlwI3w_Q7ZVzH1dRkKAjDJuTO9Bk8w5R-JYTPrUztCKmFRaW8ET1r9LI84kNLvJaUTj7EcIzmgskaboLYqeQ1tteeAd75wNMykd9rT0CmG3w4y7waXhtOUKRyyqzZTbNr3FkmFJYqR1KeS/s1600/+datanal+log_transformation_and_fitting_models-1+Tuft+2013-02-10.JPG" height="266" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
This is a very clear and didactic explanation of the importance of logarithmic transformation that anyone on doing data analysis in natural sciences or epidemiology must read.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
And a very important point is to raise the point that regression analysis of a model DOES NOT TEST the relationship but SHOWS the proportionality GIVEN THE MODEL BEING TRUE </div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgbB-Yhvxr3o-LLKaCaDHN7zNtehgSSzL7_W0pvnTc6tvt_Cb433MkrqRu8uJDCY3iogHuodoysjNxD-Sr11nuexOGiMWw9UjscvksxEGX5uwSBtJtZa44bQuURPsWyYPCRb6_QcMpjg4yC/s1600/+datanal+log_transformation_and_fitting_models-2+Tuft+2013-02-10.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgbB-Yhvxr3o-LLKaCaDHN7zNtehgSSzL7_W0pvnTc6tvt_Cb433MkrqRu8uJDCY3iogHuodoysjNxD-Sr11nuexOGiMWw9UjscvksxEGX5uwSBtJtZa44bQuURPsWyYPCRb6_QcMpjg4yC/s1600/+datanal+log_transformation_and_fitting_models-2+Tuft+2013-02-10.JPG" height="211" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
The end part of this section has a bit more of mathematics that some biologist probably have already forgotten but it is worthy to read it anyway.<br />
<br />
I truly recommend reading this even it is a very old book (ed. 1976).<br />
<br />
Final note: Remember to add 1 to your data before log transform in order to avoid log(0). Don't do that if you have negative number ;-). Other option is to add a small quantity to all your 0sPablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com0tag:blogger.com,1999:blog-7760425276056032223.post-35777675010284101572013-02-09T04:54:00.001-08:002013-02-09T09:47:59.850-08:00BioPerl is thinking about to be more practical and adaptative<br />
There has been a good number of BioPerl threads in the mailing list [0] last week about how to make BioPerl more fitted to the current times.<br />
<br />
[0] <a href="http://thread.gmane.org/gmane.comp.lang.perl.bio.general">http://thread.gmane.org/gmane.comp.lang.perl.bio.general</a><br />
<br />
I like the phrase of George Hartzell about being able to move forward because we need to support Perl 5.8<br />
<br />
<blockquote class="tr_bq">
But why should the all-volunteer BioPerl community be stuck supporting<br />
code from 12 years ago because it's cost effective for someone else to<br />
avoid spending *their* $/time/people to stay up to date.</blockquote>
<br />
And the links to the discussion:<br />
Next BioPerl release : <a href="http://thread.gmane.org/gmane.comp.lang.perl.bio.general/26348">http://thread.gmane.org/gmane.comp.lang.perl.bio.general/26348</a><br />
dependencies on perl version : <a href="http://thread.gmane.org/gmane.comp.lang.perl.bio.general/26344">http://thread.gmane.org/gmane.comp.lang.perl.bio.general/26344</a><br />
BioPerl future : <a href="http://thread.gmane.org/gmane.comp.lang.perl.bio.general/26394">http://thread.gmane.org/gmane.comp.lang.perl.bio.general/26394</a><br />
removing packages from bioperl-live: <a href="http://thread.gmane.org/gmane.comp.lang.perl.bio.general/26341">http://thread.gmane.org/gmane.comp.lang.perl.bio.general/26341</a>Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com1tag:blogger.com,1999:blog-7760425276056032223.post-83899643349575023802013-01-25T07:55:00.001-08:002013-01-25T07:55:58.551-08:00my emacs24 needs (require 'appt) in .emacs
I have updated to emacs 24 but now my .emacs file that was working OK with 23 is not working anymore
<div class="code">
<pre>
Warning (initialization): An error occurred while loading `/home/pmg/.emacs':
Symbol's function definition is void: appt-make-list
</pre>
</div>
I had this entry in my .emacs
<div class="code">
<pre>
(add-hook 'diary-hook 'appt-make-list)
</pre>
</div>
And from this <a href="http://thread.gmane.org/gmane.emacs.orgmode/47278/focus=47350">mailing list entry</a> I found the solution.
I need to add <b>(require 'appt)</b>
<div class="code">
<pre>
;; from the emacs wiki -> calendar and diary
;; Mx-calendar -> d (show day) i d (insert task)
(require 'appt)
(add-hook 'diary-hook 'appt-make-list)
(diary)
</pre>
</div>
And now all works OK.
Why I did not needed before? I don't have a cluePablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com0tag:blogger.com,1999:blog-7760425276056032223.post-69282617793341712462012-11-17T12:19:00.001-08:002012-11-17T15:53:47.892-08:00Nice positional sequencing hack in an illumina flowcell by Shendure's labAnother interesting paper from the Shendure's lab<br />
<br />
<br />
<br />
<div class='box'>
<div class="cit" style="background-color: white; border: blue,1px; font-family: arial, helvetica, clean, sans-serif; font-size: 0.8465em; font: inherit; line-height: 1.45em; margin: 0px; padding: 0px; vertical-align: baseline;">
<a abstractlink="yes" alsec="jour" alterm="Proc Natl Acad Sci U S A." aria-expanded="false" href="http://www.ncbi.nlm.nih.gov/pubmed/23112150#" role="button" style="border: 0px; color: #660066; font-size: 11px; font: inherit; margin: 0px; padding: 0px; vertical-align: baseline;" title="Proceedings of the National Academy of Sciences of the United States of America.">Proc Natl Acad Sci U S A.</a> 2012 Nov 13;109(46):18749-54. doi: 10.1073/pnas.1202680109. Epub 2012 Oct 29.</div>
<h1 style="background-color: white; border: 0px; font-family: arial, helvetica, clean, sans-serif; font-size: 1.231em; font: inherit; line-height: 1.125em; margin: 0.375em 0px; padding: 0px; vertical-align: baseline;">
Capturing native long-range contiguity by in situ library construction and optical sequencing.</h1>
<div class="auths" style="background-color: white; border: 0px; font-family: arial, helvetica, clean, sans-serif; font-size: 0.923em; font: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">
<a href="http://www.ncbi.nlm.nih.gov/pubmed?term=Schwartz%20JJ%5BAuthor%5D&cauthor=true&cauthor_uid=23112150" style="border: 0px; color: #660066; font-size: 12px; font: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Schwartz JJ</a>, <a href="http://www.ncbi.nlm.nih.gov/pubmed?term=Lee%20C%5BAuthor%5D&cauthor=true&cauthor_uid=23112150" style="border: 0px; color: #660066; font-size: 12px; font: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Lee C</a>, <a href="http://www.ncbi.nlm.nih.gov/pubmed?term=Hiatt%20JB%5BAuthor%5D&cauthor=true&cauthor_uid=23112150" style="border: 0px; color: #660066; font-size: 12px; font: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Hiatt JB</a>, <a href="http://www.ncbi.nlm.nih.gov/pubmed?term=Adey%20A%5BAuthor%5D&cauthor=true&cauthor_uid=23112150" style="border: 0px; color: #660066; font-size: 12px; font: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Adey A</a>, <a href="http://www.ncbi.nlm.nih.gov/pubmed?term=Shendure%20J%5BAuthor%5D&cauthor=true&cauthor_uid=23112150" style="border: 0px; color: #660066; font-size: 12px; font: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Shendure J</a>.</div>
<div class="aff" style="background-color: white; border: 0px; font-family: arial, helvetica, clean, sans-serif; font-size: 0.8465em; font: inherit; line-height: 1.0915em; margin: 0px; padding: 0px; vertical-align: baseline;">
<h3 class="label" style="border: 0px; color: #724128; font-size: 1.0769em; font: inherit; height: 1px; left: -10000px; line-height: 1.2857; margin: 1.2856em 0px 0.6428em; overflow: hidden; padding: 0px; position: absolute; top: auto; vertical-align: baseline; width: 1px;">
Source</h3>
<div style="border: 0px; font-size: 11px; font: inherit; margin-bottom: 0.5em; margin-top: 0.5em; padding: 0px; vertical-align: baseline;">
Department of Genome Sciences, University of Washington, Seattle, WA 98195.</div>
</div>
<div class="abstr" style="background-color: white; border: 0px; font-family: arial, helvetica, clean, sans-serif; font-size: 13px; font: inherit; line-height: 17px; margin: 1.2em auto auto; padding: 0px; vertical-align: baseline;">
<h3 style="border: 0px; color: #985735; font-size: 1.0769em; font: inherit; line-height: 1.2857; margin: 0px; padding: 0px; vertical-align: baseline;">
Abstract</h3>
<div style="border: 0px; font-size: 13px; font: inherit; margin-bottom: 0.5em; padding: 0px; vertical-align: baseline;">
The relatively short read lengths associated with the most cost-effective DNA sequencing technologies have limited their use in de novo genome assembly, structural variation detection, and haplotype-resolved genome sequencing. Consequently, there is a strong need for methods that capture various scales of contiguity information at a throughput commensurate with the current scale of massively parallel sequencing. We propose in situ library construction and optical sequencing on the flow cells of currently available massively parallel sequencing platforms as an efficient means of capturing both contiguity information and primary sequence with a single technology. In this proof-of-concept study, we demonstrate basic feasibility by generating >30,000 Escherichia coli paired-end reads separated by 1, 2, or 3 kb using in situ library construction on standard Illumina flow cells.<span style="background-color:yellow"> We also show that it is possible to stretch single molecules ranging from 3 to 8 kb on the surface of a flow cell before in situ library construction, thereby enabling the production of clusters whose physical relationship to one another on the flow cell is related to genomic distance.</span></div>
</div>
<div class="aux" style="background-color: white; border: 0px; font-family: arial, helvetica, clean, sans-serif; font-size: 13px; font: inherit; line-height: 1.4em; margin: 1.2em auto auto; padding: 0px; vertical-align: baseline;">
<div class="resc" style="border: 0px; color: #575757; font-size: 13px; font: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">
<dl class="rprtid" style="border: 0px; display: inline; font-size: 0.8465em; font: inherit; line-height: 1.4em; margin: 0px 15px 0px 0px; padding: 0px; vertical-align: baseline;">
<dt style="border: 0px; display: inline; font-size: 11px; font: inherit; margin-bottom: 0px; margin-left: 0px !important; margin-right: 0px; margin-top: 0px; padding: 0px; vertical-align: baseline; white-space: nowrap;">PMID:</dt>
<dd style="border: 0px; display: inline; font-size: 11px; font: inherit; margin: 0px; padding: 0px; vertical-align: baseline; white-space: nowrap;">23112150</dd> <dd style="border: 0px; display: inline; font-size: 11px; font: inherit; margin: 0px; padding: 0px; vertical-align: baseline; white-space: nowrap;">[PubMed - in process]</dd></dl>
</div>
</div>
</div>
<br />
<br />Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com0tag:blogger.com,1999:blog-7760425276056032223.post-73725445779440887162012-11-17T09:20:00.000-08:002012-11-17T09:20:04.268-08:00Looking for a Perl tool like Ipython notebook.<br /><br />I have just recently discovered Ipython notebooks[1] they are fantastics and like R sweave[2] they are essential for Reproducible Research.<br />
<br />I use Perl and pdl for bioinformatics research. Being pdl a scientific tool these kind of reproducible research utilities seems a perfect match.<br />
<br />
Some time ago in the bio(perl|python) communities they started to look to use a similar approach to sweave but I have not seen much progress on that. A cheap alternative (if your OS is called emacs ;-)) is to use org-mode babel[3]. I like to use babel because I almost always work under ssh connection to my servers using also screen session with emacs -nw . Babel could be a substitute for sweave, but Ipython notebook is a more advanced and interactive beast. You can see this fantastic presentation from Fernando Pérez about scientific Python[4] where he demonstrates its powers.<br />
<br />
I don't know yet any Perl tool similar for Ipython notebooks but if it exist I would like to find and use it.<br />
<br />Any comments on this topic and links would be more than wellcome.<div>
<div>
<br /><span style="font-size: x-small;">[1] <a href="http://blog.fperez.org/2012/09/blogging-with-ipython-notebook.html">http://blog.fperez.org/2012/09/blogging-with-ipython-notebook.html</a><br />[2] <a href="http://www.stat.uni-muenchen.de/~leisch/Sweave/">http://www.stat.uni-muenchen.de/~leisch/Sweave/</a><br />[3] <a href="http://orgmode.org/worg/org-contrib/babel/">http://orgmode.org/worg/org-contrib/babel/</a></span><div>
<span style="font-size: x-small;">[4] <a href="http://www.youtube.com/watch?v=F4rFuIb1Ie4">http://www.youtube.com/watch?v=F4rFuIb1Ie4</a></span><br /></div>
</div>
</div>
Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com6tag:blogger.com,1999:blog-7760425276056032223.post-55681229192839077092012-11-17T03:09:00.001-08:002012-11-17T04:47:52.378-08:00Key points summary in the GATK licence change<br />
Recently <a href="http://pablomarin-garcia.blogspot.co.uk/2012/11/gatk-20-drops-mit-licence.html">I posted about GATK change of licence</a> to a commercial one dropping MIT licence for GATK 2.0.<br />
<br />
There is a <a href="http://gatkforums.broadinstitute.org/discussion/1261/discussion-of-the-new-licensing-and-mixed-open-source-model"> page with the thread generated by the licence change at gatkforums</a>. And first of all, I am very glad to see how scientist are discussing with argumentation and opinion instead of trolling and insulting. In a informatics forum this would generate a rather unpolite flame war. I am very please that holocaust and Hitler has not been mentioned yet and seem that never will be ;-). Kudos to the community.<br />
<br />
From this thread seems that the licence change was done to prevent some companies from making money selling GATK analysis meanwhile the lab "producing" GATK is year by year, as the rest of us, desperately trying to secure economic resources to keep on going the projects in this time of big cuts on science. That is understandable but this would rise another debate: when are tools like GATK, SAMtools pipeline, tuxedo suit etc. getting mature enough to fly solo and pass from a scientific project to a software production?. When this happens, should they be funded by scientific founds or by industry founds?<br />
<br />
The key of the dabate here is more about the lack of expertise of the lab in licencing and commercial and retriction implications as the licence terms where ambiguous and not clear enough at some passages.<br />
<div>
<br /></div>
<br />
<div class="box">
<div class="box">
<pre>[dePristo reply in the blog explaining the two licences]
Hi Pepetideo,
You can share your GATK results -- that was a language slip up on our part. You can see it's clear in the license and FAQ now.
As for why, its two fold. One is to ensure that we can continue to develop and support the GATK into the future by creating a sustainable revenue source for the team. Two is that a commercial version will be able to support a large team providing tier-one support, such as long-term maintenance of specific GATK versions, which my research group simply cannot provide. Note that any commercial entity who wants to stay with GATK-Lite can go the full open source route, at the cost of foregoing premium support and access to the best possible tools.
We recognize that this is a change, and of course we are big supporters of open source software -- the vast majority of the GATK2 is open source. We considered creating a "GATK foundation" mozilla style, accepting micro donations, or even providing pay-for-services on top of the GATK but ultimately the commercial/non-commercial divide seemed the option that provides the most value to the entirely community.
Purchasing a commercial GATK2 license will give you the right to run the GATK2 within the company and share / publish / etc your results. This is what I'd think of as a standard commercial license, and most places would fit in this bucket. The example here is buying Adobe Photoshop and using it in house to manage and edit photos.
The more complex question is around third-party pipeline executors, which only take in data from others and who effectively sell the running of the GATK. Here I think there will be a separate license with specific terms, but it's something we'd like to enable. The analogy here is setting up a for-profit web portal for photo editing that backends to photoshop. A valuable activity but one not covered by the standard end-user license agreement.
</pre>
</div>
[some answers to that]
<br />
<div class="box">
<pre>TechnicalVault
August 2
From this side of the pond the Wellcome Trust Sanger Institute does have a policy on the software we develop here, it has to be open sourced and under specified licenses. This is in harmony with our policy that research funded by Wellcome Trust money must be published in an open access journal.
I can't speak for the institute itself but I have a sinking feeling this decision will spark a lot of debate. The concern this gives me and what I intend to find out about, is how this will interact with any collaborations on or contributions we with to make to GATK. UK charity law is quite tight on what kind of profit making activities charities can take part in so it may involve lawyer time.
I do understand the Broad's point of view, people are making money on software that the Broad has invested money in producing and the Broad is not getting a cut from it. Ideally the way they'd pay it forward would be to contribute testing time and improvements back, but in practice I imagine quite a few are taking a free ride. That said companies take a free ride on most of the research we do, it's just harder to make money from most of it though. This whole debate does bring the name Celera to mind though.
</pre>
</div>
<div class="box">
<pre>joshkorn
August 3
Companies pay taxes too. Some additionally indirectly support the development of tools such as the GATK through academic collaborations. I guess the way you are framing the question though, it comes down to this: taxpayers paid for the development of the GATK. Most taxpayers aren't doing genetic research. So what will benefit the taxpayers most? (Should companies be paying for the reference genome sequence? For SNP databases? Where do we draw the line?) The reason the government (taxpayers) invest in basic research is to stimulate the downstream discovery. Help us translate research into helping patients!
I even take exception to the phrase "If the time comes when they're asked to contribute back and they don't, then yes they are leeches." Who is asking to contribute back? Not the people who paid for the development in the first place (NIH, Eli & Edyth Broad, Harvard, MIT)! The GATK became widely used not only because it was good (it is), and not only because it was free (it was), but because of a huge investment from other projects (most notably the 1000 genomes project, but others as well) that got it free publicity and turned it into a de facto standard. It's hard to compete when the GATK team has earlier access to taxpayer-funded projects/data/sequencing, and guaranteed publications when these projects come out. Also, gaining market share and then raising prices sounds more capitalist than Marxist, to reference an earlier comment.
I don't mean to go on a tirade; I guess I just feel strongly about this. I definitely understand where you are coming from; I too have written popular open-source software that cost me personally plenty of time of support, and meanwhile surely helped some for-profit entities do research (I hope!) and one for-profit company in particular have success. Nonetheless, I knew it was my duty to share the software freely. Also please know that I say all this with the utmost respect for the whole GATK team (most of which I think I know--I don't think we've had the pleasure of meeting though, Geraldine). You guys are doing a great job, and it's wonderful that taxpayers have been able to fund the development of (documented, supported) academic software. I'd be happy to take everyone out for drinks after work some day and thank you personally; justifying paying for the software is difficult.
*Note: these views are my own, and do not necessarily reflect those of my company or colleagues.
</pre>
</div>
<div class="box">
<pre>Mark_DePristo
August 3
Hi all,
I want to chime in with three clarifying points:
We don't yet know the pricing scheme, but we are keenly aware of the complications of per-use licensing as TechnicalVault brings up
Overall I want everyone to tone down the moral issue surrounding commercial licensing. The discussion of moral issues, extracting of rents, leeching off taxpayers, are all counterproductive to helping understand what we have decided and the best path forward. All of this is just business, after all.
The NIH is clear that when funding basic research that the support IP is generally owned the developing institution (I'm sure there are exceptions), and this is true for software in general. The only key software restriction I know of is that the software must be made available to federal employees upon request. The reason for this policy is obvious -- it would be extremely difficult to accept the trade-off in a grants with IP ownership if you are creating high-value IP. Even federal SBIR grants spell out clearly that the government does not own any IP associated with the support. The federal granting system is to foster innovation, not to own innovation. It's a subtle difference but important for anyone creating real IP value with federal money in the US
Many users of the GATK would like a much higher level of support than the Broad institute could possibly provide, as this is off track for the Broad's mission to transform medicine through genomics. We believe that having a commercial license for the GATK will allow us to actually deliver on this superior support and continue to grow the GATK as a reliable standard for NGS analysis in the commercial sphere and beyond. Without a commercial version we simply cannot follow through on this opportunity.
We attempted to make the GATK easy for others to contribute code to, but our experiences in this area have been disappointing. Many people use the GATK for developing tools -- and we are committed to ensuring the programming framework and libraries remain MIT licensed -- we have had little contribution over 3 years to the master codebase from independent third-parties. Certainly some of our collaborators have contributed impressive tools and extensions, but again they aren't really independent. There's a good wikipedia article on the experiences of mySQL similar to this, and they release with a dual-license similar to our approach. Still though I'd like to release the source code to all of the tools -- if we can find a way consistent with the commercial license -- for transparency to the community and to allow others to contribute, in so far as they like.
-- Mark A. DePristo, Ph.D. Co-Director, Medical and Population Genetics Broad Institute of MIT and Harvard
</pre>
</div>
But people still confused 2 month later
<br />
<div class="box">
<pre>TechnicalVault Posts
October 17
Hi Mark, I have a couple of questions stemming from the FAQ posted by your new commercial partners:
Regarding, "Why not stay with GATKLite?" According to the FAQ at your new partner's site "Broad has indicated that GATK-lite tools will soon be obsolete, and it plans to stop supporting the tools by the end of 2012." Can you confirm exactly what this means please? Is it all of GATK-lite which will be dropped or just tools which have been replaced by new ones from GATK2?
"Use by a not-for-profit organization to generate revenue requires a commercial license", can you clarify what this means? For example providing sequencing services to other academic institutions generates revenue, however it is usually done at cost so does not generate profit.
If a not-for-profit was interested in buying support, but not in buying a commercial license is there an option for this? Who would it be with?
Finally who will be the final arbiter of usage terms? Does that remain with the Broad or have you signed enforcement over to your partners?
Thank you for all your hard work
Post edited by TechnicalVault on October 17
</pre>
</div>
<div class="box">
<pre>Geraldine_VdAuwera
October 17
Hi Martin,
CORRECTION: The wording in the FAQ was incorrect due to a miscommunication. We will in fact continue providing support for GATK-Lite tools indefinitely, and although we will eventually stop providing a separate build (jar file), the GATK-Lite codebase will remain publicly available on our open source GitHub repository. In addition, tools from GATK 2 will be migrated into the GATK-Lite codebase over time.
and 3. Please direct these questions to our partner, Appistry. They will be able to tell you based on your specific circumstances. They have a contact form that you can use, and within a few days they will also have a discussion forum that you can use for this purpose.
See above.
I believe Mark will answer that for you, or direct you to Issi Rozen here at the Broad, if you want an answer from our side. Otherwise I expect the Appistry people should be able to answer that as well.
</pre>
</div>
<div class="box">
<pre></pre>
</div>
<div class="box">
<pre></pre>
</div>
<div class="box">
<pre></pre>
</div>
</div>
Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com0tag:blogger.com,1999:blog-7760425276056032223.post-26207703942364868762012-11-16T20:27:00.000-08:002012-11-17T04:44:22.250-08:00GATK 2.0 drops MIT licence<br />
Sadly Broad Institute's great contribution to new generation sequencing NGS, <a href="http://gatkforums.broadinstitute.org/discussion/17/gatk-2-0-announcement">GATK, is not open source anymore</a>.<br />
<br />
<blockquote>
GATK 2.0 is moving to a mixed open/closed-source model<br />
The complete GATK 2.0 suite will be distributed as a binary only,
without source code for the newest tools. We plan to release the source
code for these tools, but its unclear the timeframe for this.</blockquote>
Personally I dislike this movement. If reason is money there are successful business models build around open source software without the need of leaving the open source community.<br />
<br />
Doing this sort of things is a sacrilege for many of the members of our community fighting day and night for the #OA. I can see RMS deeply depress and crying in a corner every time such a thing happens. Whatever the reason behind this change is, presumably it has been a long meditated one. So I would expect a better explanation of why this change was needed, why keep being an open source software was impeding this goal, and what is more important, the thing that annoys me more, and denotes poor strategic plan for such a big change "We plan to release the source code for these tools, but its unclear the timeframe for this". Hummm, this seems to me that commitment for open source has vanished here. Even if you believe in OS, some times you are forced to make concessions to close software but I would expect to leave clear note when and how your source code would be open again.<br />
<br />
How come we are going to lead a war for open access journals and against patents but want to make money of our software. This seems a deadly friendly fire to me until I read a good detailed explanation of the reasons behind.<br />
<br />
It is a good thing to have samtools around as a backup.<br />
<br />
<div class="box">
== follow up ==<br />
http://pablomarin-garcia.blogspot.co.uk/2012/11/key-points-summary-in-gatk-licence.html
<br /></div>Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com0tag:blogger.com,1999:blog-7760425276056032223.post-37795903928240259842012-11-15T17:31:00.001-08:002012-11-15T17:31:36.223-08:00bioperl popularity (measured by searches) going down day by day.<br />
<div>
According to the following figure bioperl is loosing all its appealing day by day. One critique to this plot is that big bio projects still are in perl like Ensembl, biopieces etc, but not shown here. The pity is that R and biophyton have very good tools/pipelines for New Generation Sequencing and bioperl or other perl bio projects don't.</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="http://www.google.com/trends/explore#q=biopython,%20bioperl,%20bioconductor">http://www.google.com/trends/explore#q=biopython,%20bioperl,%20bioconductor</a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhV6FS148xkvBARdT8kexDrJfYoaNMg2_hvbeBxZqq25QdPh2-EAYyPHYM5Mk-80HP-hvq0hS9TmVnCntQZ0c-VCUh0uaYEznaV8113B_UK8y7fqjT28babuSEU8XKDcFQ4cZC9J6hySjHf/s1600/+bioinfo+google-trends-bioperl++2012-11-14.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhV6FS148xkvBARdT8kexDrJfYoaNMg2_hvbeBxZqq25QdPh2-EAYyPHYM5Mk-80HP-hvq0hS9TmVnCntQZ0c-VCUh0uaYEznaV8113B_UK8y7fqjT28babuSEU8XKDcFQ4cZC9J6hySjHf/s1600/+bioinfo+google-trends-bioperl++2012-11-14.PNG" height="240" width="640" /></a></div>
<div>
<br /></div>
Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com1tag:blogger.com,1999:blog-7760425276056032223.post-77099156560971910942012-11-07T06:56:00.001-08:002012-11-07T06:56:13.120-08:00how to use R introspection for obtaining a data.frame from an objectA very helpful post from Charlot Wickham from the ggplot2 mailing list about how to find out which methods to use to extract a data.frame with the data from an object.<br />
<div>
<br />
Question:
<br />
<div class="box">
<div class="gE iv gt" style="background-color: white; color: #222222; cursor: pointer; font-family: arial, sans-serif; font-size: 13px; padding: 10px 0px 3px;">
<table cellpadding="0" class="cf gJ" style="border-collapse: collapse; margin-top: 0px; width: auto;"><tbody>
<tr class="acZ" style="height: 16px;"><td class="gF gK" style="margin: 0px; padding-right: 8px; padding-top: 0px; vertical-align: top; white-space: nowrap; width: 575px;"><table cellpadding="0" class="cf ix" style="border-collapse: collapse; table-layout: fixed; width: 575px;"><tbody>
<tr><td style="margin: 0px;"><div class="iw" style="display: inline-block; max-width: 92%; overflow: hidden; white-space: nowrap;">
<span class="gD" email="lecomte.jeanbaptiste@gmail.com" name="Lecomte Jean-Baptiste" style="color: #222222; display: inline; font-size: 13px; font-weight: bold; vertical-align: top;">Lecomte Jean-Baptiste</span></div>
</td></tr>
</tbody></table>
</td><td class="gH" style="color: #222222; margin: 0px; text-align: right; vertical-align: top; white-space: nowrap;"><div class="gK" style="padding-right: 2px; padding-top: 0px;">
<span alt="Tue, Nov 6, 2012 at 6:16 PM" class="g3" id=":153" style="margin-right: 3px; vertical-align: top;" title="Tue, Nov 6, 2012 at 6:16 PM">6:16 PM (20 hours ago)</span></div>
</td>
</tr>
<tr class="acZ xD" style="height: 16px;"><td colspan="3" style="margin: 0px;"><table cellpadding="0" class="cf adz" style="border-collapse: collapse; table-layout: fixed; white-space: nowrap; width: 747px;"><tbody>
<tr><td class="ady" style="margin: 0px; overflow: hidden;"><div class="iw ajw" style="display: inline-block; max-width: 92%; overflow: hidden;">
<span class="hb" style="color: #777777; vertical-align: top;">to <span class="g2" email="ggplot2@googlegroups.com" name="ggplot2" style="vertical-align: top;">ggplot2</span></span></div>
<div aria-haspopup="true" class="ajy" style="display: inline-block; margin-left: 5px; vertical-align: top;">
</div>
</td></tr>
</tbody></table>
</td></tr>
</tbody></table>
</div>
<div class="utdU2e" style="background-color: white; color: #222222; font-family: arial, sans-serif;">
</div>
<div class="tx78Ic" style="background-color: white; color: #222222; font-family: arial, sans-serif;">
</div>
<div class="QqXVeb" style="background-color: white; color: #222222; font-family: arial, sans-serif;">
</div>
<div id=":om" style="background-color: white; color: #222222; font-family: arial, sans-serif;" tabindex="-1">
</div>
<div class="ii gt adP adO" id=":151" style="background-color: white; color: #222222; direction: ltr; font-family: arial, sans-serif; font-size: 13px; margin: 5px 15px 0px 0px; padding-bottom: 5px; position: relative; z-index: 2;">
<div id=":150">
Dear all,<br />
<br />
I'm trying to plot with ggplot2 the result of the function variofit of the geoR package.<br />
It's quite simple with the basic plot :<br />
<br />
<span style="color: #666666;">vario100 <- variog(s100, max.dist=1)<br />ini.vals <- expand.grid(seq(0,1,l=5), seq(0,1,l=5))<br />ols <- variofit(vario100, ini=ini.vals, fix.nug=TRUE, wei="equal")<br />summary(ols)<br />wls <- variofit(vario100, ini=ini.vals, fix.nug=TRUE)<br />summary(wls)<br />plot(vario100)<br />lines(wls)<br />lines(ols, lty=2)</span><br />
<br />
I can plot the points of the empirical variogram, but I can't plot the line representing the fitted variogram with ggplot2<br />
<br />
<span style="color: #666666;">df_vario<-data.frame(u=<wbr></wbr>vario100$u,v=vario100$v)<br /><br />qplot(u,v,data=df_vario)<br /><br />qplot(u,v,data=df_vario)+geom_<wbr></wbr>line(wls)</span><br />
I have make a quick search on both ggplot2 and R-sig-Geo user list without finding any solutions.<br />
I will appreciate any advice.<br />
Thanks<br />
<br />
Jean-Baptiste Lecomte</div>
</div>
</div>
</div>
Answer:
<br />
<div class="box">
<br />
<div class="gE iv gt" style="background-color: white; color: #222222; cursor: auto; font-family: arial, sans-serif; font-size: 13px; padding: 10px 0px 3px;">
<table cellpadding="0" class="cf gJ" style="border-collapse: collapse; margin-top: 0px; width: auto;"><tbody>
<tr class="acZ" style="height: 16px;"><td class="gF gK" style="margin: 0px; padding-right: 8px; padding-top: 0px; vertical-align: top; white-space: nowrap; width: 568px;"><table cellpadding="0" class="cf ix" style="border-collapse: collapse; table-layout: fixed; width: 568px;"><tbody>
<tr><td style="margin: 0px;"><div class="iw" style="display: inline-block; max-width: 92%; overflow: hidden; white-space: nowrap;">
<span class="gD" email="cwickham@gmail.com" name="Charlotte Wickham" style="color: #222222; display: inline; font-size: 13px; font-weight: bold; vertical-align: top;">Charlotte Wickham</span></div>
</td></tr>
</tbody></table>
</td><td class="gH" style="color: #222222; margin: 0px; text-align: right; vertical-align: top; white-space: nowrap;"><div class="gK" style="padding-right: 2px; padding-top: 0px;">
<span alt="Tue, Nov 6, 2012 at 11:08 PM" class="g3" id=":og" style="margin-right: 3px; vertical-align: top;" title="Tue, Nov 6, 2012 at 11:08 PM">11:08 PM (15 hours ago)</span><div aria-checked="false" aria-label="Starred" class="zd" role="checkbox" style="display: inline-block; height: 20px; outline: 0px;" tabindex="-1">
<span class="T-KT" style="display: inline-block; height: 19px; margin: -4px 0px; padding: 2px; text-align: center; width: 19px;"><img alt="" class="f T-KT-JX" src="https://mail.google.com/mail/images/cleardot.gif" style="margin-top: 0px; vertical-align: top;" /></span></div>
</div>
</td><td class="gH" style="color: #222222; margin: 0px; text-align: right; vertical-align: top; white-space: nowrap;"><br /></td><td class="gH acX" rowspan="2" style="color: #222222; margin: 0px; text-align: right; vertical-align: top; white-space: nowrap;"></td></tr>
<tr class="acZ xD" style="height: 16px;"><td colspan="3" style="margin: 0px;"><table cellpadding="0" class="cf adz" style="border-collapse: collapse; table-layout: fixed; white-space: nowrap; width: 747px;"><tbody>
<tr><td class="ady" style="margin: 0px; overflow: hidden;"><div class="iw ajw" style="display: inline-block; max-width: 92%; overflow: hidden;">
<span class="hb" style="color: #777777; vertical-align: top;">to <span class="g2" email="lecomte.jeanbaptiste@gmail.com" name="Lecomte" style="vertical-align: top;">Lecomte</span>, <span class="g2" email="ggplot2@googlegroups.com" name="ggplot2" style="vertical-align: top;">ggplot2</span></span></div>
<div aria-haspopup="true" class="ajy" style="display: inline-block; margin-left: 5px; vertical-align: top;">
</div>
</td></tr>
</tbody></table>
</td></tr>
</tbody></table>
</div>
<div class="utdU2e" style="background-color: white; color: #222222; font-family: arial, sans-serif;">
</div>
<div class="tx78Ic" style="background-color: white; color: #222222; font-family: arial, sans-serif;">
</div>
<div class="QqXVeb" style="background-color: white; color: #222222; font-family: arial, sans-serif;">
</div>
<div id=":o5" style="background-color: white; color: #222222; font-family: arial, sans-serif;" tabindex="-1">
</div>
<div class="ii gt adP adO" id=":oi" style="background-color: white; color: #222222; direction: ltr; font-family: arial, sans-serif; font-size: 13px; margin: 5px 15px 0px 0px; padding-bottom: 5px; position: relative; z-index: 2;">
<div id=":oj">
<div>
Hi Jean-Baptiste,</div>
<div>
<br /></div>
<div>
There are a couple of problems with geom_line(wls). Firstly ols and wls are special objects that ggplot knows nothing about, and secondly you are passing them in as the first argument where geom_line expects a mapping. A general solution for this type of problem is to write a function that takes the special type of object you are interested in as an argument, and outputs a data.frame you can use ggplot to plot.</div>
<div>
<br /></div>
<div>
You know lines(wls) must at some point calculate enough info to draw the line you require, it's just a matter of finding out where.</div>
<div>
<br /></div>
This first step is too figure out what is happening in lines(ols):<div>
<div>
> class(ols)</div>
<div>
[1] "variomodel" "variofit" </div>
</div>
<div>
<br /></div>
<div>
Tells us ols has class "variomodel", so our best bet is to look at lines.variomodel:</div>
<div>
<br /></div>
<div>
<div>
> ?lines.variomodel</div>
<div>
> lines.variomodel</div>
<div>
function (x, ...) </div>
<div>
{</div>
<div>
UseMethod("lines.variomodel")</div>
<div>
}</div>
<div>
<environment: namespace:geoR></div>
</div>
<div>
<br /></div>
<div>
Not very helpful...looks like lines.variogram is a generic function, we need to figure out which method is being called. </div>
<div>
<br /></div>
<div>
<div>
> methods("lines.variomodel")</div>
<div>
[1] lines.variomodel.default* lines.variomodel.grf* </div>
<div>
[3] lines.variomodel.krige.bayes* lines.variomodel.likGRF* </div>
<div>
[5] lines.variomodel.variofit* </div>
<div>
<br /></div>
<div>
Non-visible functions are asterisked</div>
</div>
<div>
<br /></div>
<div>
Look like lines.variomodel.variofit is our candidate:</div>
<div>
getAnywhere("lines.variomodel.<wbr></wbr>variofit")</div>
<div>
<br /></div>
<div>
will spit out the function, and it looks like most of it is calculating the fitted line to be used in the call to the function curve. One way to get ggplot to plot the fitted line would be to reuse that code but instead of calling curve, outputting a data.frame that ggplot can use. I put my attempt here: <a href="https://gist.github.com/4027851" style="color: #1155cc;" target="_blank">https://gist.github.com/<wbr></wbr>4027851</a></div>
<div>
<br /></div>
<div>
Source that in and try:</div>
<div>
<span style="color: #666666;">qplot(u,v,data=df_vario) +</span></div>
<div>
geom_line(aes(x, fit), data = fitted_variofit(ols)) +</div>
<div>
geom_line(aes(x, fit), data = fitted_variofit(wls), linetype = "dashed")</div>
<div>
<br /></div>
<div>
Hope that helps,</div>
<div>
<br /></div>
<div>
Charlotte</div>
</div>
</div>
</div>
Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com0tag:blogger.com,1999:blog-7760425276056032223.post-89868397635366163102012-10-19T07:24:00.000-07:002012-10-19T07:24:16.704-07:00icon fonts, a nice way of dealing with icon/images as tex<h2>
Icon Fonts</h2>
<div>
A nice way of plotting icons or small graphic elements as text</div>
<div>
<br /></div>
<a href="http://css-tricks.com/examples/IconFont">http://css-tricks.com/examples/IconFont</a><br />
<a href="http://css-tricks.com/flat-icons-icon-fonts/">http://css-tricks.com/flat-icons-icon-fonts/</a>Pablo Marin-Garciahttp://www.blogger.com/profile/17942399687345864870noreply@blogger.com0