Thursday, February 13, 2014

Git blame statistics

Well, I decided to see how much I contributed to my current project and this is the solution I found, using git:
git ls-tree -r HEAD|sed -re 's/^.{53}//'|while read filename; do file "$filename"; done|grep -E ': .*text'|sed -r -e 's/: .*//'|while read filename; do git blame -w "$filename"; done|sed -r -e 's/.*\((.*)[0-9]{4}-[0-9]{2}-[0-9]{2} .*/\1/' -e 's/ +$//'|sort|uniq -c &>stats
Taken from here with some minor changes 

Tuesday, July 23, 2013

HOWTO Migrate from SVN to GIT. A brief guide

I post it here as it would be pity to loose.

Read this first:
http://john.albin.net/git/convert-subversion-to-git


# create authors mapping file
cd
svn log -q | awk -F '|' '/^r/ {sub("^ ", "", $2); sub(" $", "", $2); print $2" = "$2" <"$2">"}' | sort -u > authors-transform.txt

# edit authors-transform.txt
# user name format:
# svn_user = git_user

# clone SVN repository into Git's
git svn clone -s https://jlwv.svn.beanstalkapp.com/ -A /authors-transform.txt --stdlayout

# push repository to a bare git repository
git init --bare
cd
git symbolic-ref HEAD refs/heads/trunk

cd
git remote add bare
git config remote.bare.push 'refs/remotes/*:refs/heads/*'
git push bare

# rename “trunk” branch to “master”
cd
git branch -m trunk master

# clean up branches and tags
git for-each-ref --format='%(refname)' refs/heads/tags |
cut -d / -f 4 |
while read ref; do
git tag "$ref" "refs/heads/tags/$ref";
git branch -D "tags/$ref";
done

# GitHub specific:

git remote add origin https://github.com/
git push -u origin master
# git push origin branch1 branch2 tag1 tag2... for each branch/tag
# do not use --all/--tags switch, otherwise will have to deal with a lot of crap

Friday, June 12, 2009

XFCE, or KDE4 is not my choice

After struggling with KDE4 and it's bugs (I wrote about that in my last post, in Russian) I had a dilemma - continue using so loved window manager (I started with KDE1) or find an alternative that would just work, run faster, probably be not so reach for features, but still offer me nice look and feel.

Gnome


On of the most popular window managers ever. I tried it years ago but dislike - there are too many panels, they occupy space on top and bottom of desktop. Trying to combine everything on a single panel takes too much to spend on such a stupid thing.

LXDE


There were many advices to try this window manager out, and at first glance it looks great. But after X session restart the panel started blinking so I had not other choice killing X server. My plan was to find a replacement for KDE, fixing bugs was out of scope. Sorry, next time maybe...

XFCE


Well, looks good but with issues - multimedia buttons do not work, seems not many settings (after KDE of course). But wait, I need a panel, multiple desktops, and nice look and feel. Want it be as cool as possible, but run fast, really fast. Will it?
Now we will pass through configuration steps and check whether result is what we expect. I can say I'm just happy with it :)
XFCE offers configuration tool that allows configure themes, keyboard layout, shortcuts, window and mouse behavior and so on. Let's try to configure them to look and feel like we used to. I skip many steps like looking through a list of window themes and choosing the best one, to save our time.





Autostarting application


This is one of the most important features KDE provides. Many of window managers (including Gnome?) do not save which applications were started in the session and run minimized into tray bar almost all the time. Applications like skype are good targets to be started on session startup, but if they don't itt looks annoying.
We may configure all the applications to start automatically on "Application Autostart" page. For example, skype is what I have on my laptop.



Shortcuts


KDE has many well defined shortcuts that we want to have on XFDE too. Unfortunately on KDE3.5 on Debian I had some multimedia keys, like Mute, unrecognized so extra work with xmodmap was required. In XFCE the issue does not exist so will just assign them to applications/actions.



On "Application Shortcuts" page of "Keyboard" configuration tool I assigned Amarok to be launched on "Music" button press, Opera browser on "WWW", tvtime on "Video":
amarok <-> XF86Music
opera <-> XF86WWW
tvtime <-> XF86Video


Volume and mute control keys presses handled with amixer application. At first I thought popup window with volume % bar would be great but later changed my mind - when control volume/mute from TV tuner remote so do not get bored with popups. Also XFCE mixer plugin handles mixer control events properly and changes icon in system tray accordingly. amixer has many commands and options, so probably you will want to change some:
amixer -q -c 1 sset 'Master',0 10%- <-> XF86AudioLowerVolume
amixer -q -c 1 sset 'Master',0 10%+ <-> XF86AudioRaiseVolume
amixer -q -c 1 sset 'Master',0 toggle <-> XF86AudioMute


Launch application


The applet behavior at first glance looks too simple - there is no autocompletion by executable file name, instead it saves command history and gives a choice from it. Simple, but works. Also, like Gnome, it fails running applications, like Synaptic, that require root privileges. I'm not sure how KDE handles this, even for Gtk applications, but it does. So looking for a command in main menu is the only way if you do not want to play with $DISPLAY and xhost.

Window decoration


Compiz ... uses 3D graphics hardware to create fast compositing desktop effects for window management (thanks to Wikipedia :) ). It creates really cool effects which combined with XFCE give you a replacement to KDE and it's graphic features (btw Compiz can be used with KDE as well).
To configure Compiz use configuration settings manager:
ccsm




Just select effects you like and then launch Compiz daemon:
compiz --replace

Add this command to "Application Autostart" so next time you login Compiz is launched automatically.







I should say that THERE IS life out of KDE, and if you are tired dealing with bugs, lost settings, and slow UI, give XFCE a chance.

More official screenshots here

Sunday, June 7, 2009

KDE4 fornever

Был такой интересный фильм Memento (рекомендую кто не видел), так вот события в фильме разворачиваются с конца. Для тек, кому лень читать, скажу сразу - Run baby, run! Не смейте устанавливать KDE4! Не меняйте 3.5 (или особенно если вы на другом менеджере окон) на это издевательство. Я предупредил. Ниже всего-лишь краткий список проблем, багов, которые были обнаружены после обновления Debian Etch на Squeeze, в течение дня. Совершенствование продукта - вещь необходимая, но путь, выбранный разработчиками, кажется ммм... неправильным что ли. Вспоминаю старые добрые времена KDE 1, 2, 3, когда обновление приносило радость, а не убитые нервные клетки и кучу вопросов вроде "Ну и, что дальше?".


Итак, визуальные дефекты. Надо признать, эта проблема пропала после обновления на последнее ядро (2.6.29.4 с 2.6.23.12), соответственно с пересборкой новых драйверов от nVidia, после чего появилось множество 3D эффектов. Но мальчик то все равно был!


Еще один баг в KDE4 - после установки курсора мыши в KDE Classic, DMZ или любой другой Oxygen, настройка применяется полностью или частично к некоторым окнам, после перезапуска KDE сбрасывается в Oxygen Black. Раздражает - не то слово. В Gnome создается впечатление, что Qt приложения используют курсор установленный в KDE. Называется не хочешь KDE - попробуй в других менеджерах поработать. Ну где же альтернатива (к этому еще вернусь)?


Мультимедиа кнопки в KDE не работают, кроме разве что управления звуком. Что интересно, регулировка звука работает на тот канал, на который настроена. Т.е. звук с TVTime может никак не регулироваться, кроме как из самого TVTime. Темная история, и вероятно, еще не оконченная. При попытке привязать кнопки вроде WWW к запуску приложений имеем баг в Qt.

Была еще масса приколов с опциями - настраивать черный текст на белом фоне в konsole приходилось раз 10 точно. Управление его профайлами - высший пилотаж, когда при 2 профайлов ты видишь 4, и какой же из них активный не понять. Если бы количество нервных клеток было неограничено, их падеж составил бы примерно число Гугл.


Еще хотелось бы покритиковать такие себе "усовершенствования", как главное меню и окно завершения работы в частности. Ну почему столько много пунктов в меню, и всегда один и тот же диалог "Жду XXX секунд..."? В 3.5 меню выглядело более продуманным что ли, здесь же навигация без мыши словно битье головой об стену. Диалог System settings переносит нас в эпоху темного средневековья и инквизиции, в смысле Windows 3.1-like. Найти нужную опцию - непростая задача. Разделение на вкладки главных и расширенных настроек - вершина деградации в разработке пользовательского интерфейса. Desktop folders - вообще непонятно что. Кому в голову могло прийти группировать кучу ярлыков в папки, да еще обведенные этими идиотскими рамочками, портящими обои? Поворот фолдера градусов на 30 вовсе не кажется необходимой опцией. Поскольку рамка имеет границы, то становится непонятно, а зачем же тогда эта всплывающая панель с 3 кнопками?

Для тех, кто все же выбрал путь воина, стоит уделить внимание обновлению того, что могло устареть. Я не про железо, но например, запуск KDE обрывался где-то после ввода пароля в gdm/kdm с появлением окна загрузки, и перед стартом KDE. Что-то вроде could not connect to ksmserver, хотя каждый раз по-разному. Проблема решилась обновлением конфигурации X (/etc/X11/xorg.conf). То же с lirc - irrecord не читал из девайса, пока не пересобрал ядро, с самим lirc.

Итог мучений

С одной стороны, команда разработчиков приклала массу усилий для усовершенствования и развития проекта, за что им, несомненно, спасибо, но такие проблемы в версии 4.2 - это печально... Ищу альтернативу.

Saturday, May 30, 2009

Configuring external monitor on Debian laptop

Here I'll try to describe how external monitor can be used on laptop in mirroring mode.

Hardware details



Laptop supports maximum resolution 1280x800.
Monitor SCALEOVIEW D22W-1 maximum resolution is 1680x1050.

X configuration



Here is /etc/X11/xorg.conf configuration adopted for HP6730b internal
display and external monitor:
Section "InputDevice"
Identifier "Generic Keyboard"
Driver "kbd"
Option "XkbRules" "xorg"
Option "XkbModel" "pc105"
EndSection

Section "InputDevice"
Identifier "Configured Mouse"
Driver "mouse"
EndSection

Section "Device"
Identifier "Primary Video Device"
# Intel driver
Driver "intel"
Option "MergedFB" "true"
# These modes specify mapping of internal monitor resolution to external
# monitor resolution, as list of two pairs (X by Y) separated with -
Option "MetaModes" "1024x768-1680x1050 1024x768-1280x1024 1024x768-1024x768 1024x768-800x600"
Option "MergedDPI" "100 100"
EndSection

Section "Device"
Identifier "Secondary Video Device"
Screen 1
EndSection

Section "Monitor"
Identifier "Laptop Monitor"
EndSection

Section "Monitor"
Identifier "External Monitor"
EndSection

Section "Screen"
Identifier "Internal Screen"
Device "Primary Video Device"
Monitor "Laptop Monitor"
DefaultDepth 24
SubSection "Display"
Depth 24
# Resolutions supported by laptop display
Modes "1024x768" "800x600"
EndSubSection
EndSection

Section "Screen"
Identifier "External Screen"
Device "Secondary Video Device"
Monitor "External Monitor"
DefaultDepth 24
SubSection "Display"
Depth 24
# Resolutions supported by external display
Modes "1680x1050" "1600x1200" "1280x1024" "1024x768" "800x600"
EndSubSection
EndSection

Section "ServerLayout"
Identifier "Default Layout"
Screen 0 "Internal Screen"
InputDevice "Generic Keyboard"
EndSection


Please notice that driver set to "intel" - generic driver does not work with VGA output.

Switching resolution



To switch to external monitor with (probably) higher
resolution, create desktop.sh script in /usr/local/bin with following content:

xrandr --output LVDS --mode 1024x768 --set BACKLIGHT_CONTROL legacy --output VGA --mode 1680x1050
# This will completely turn laptop display off, commented out since we control backlight instead
# xrandr --output LVDS --off
# This will set backlight to 10%
xbacklight -set 10


To switch to smaller resolution (that laptop display supports), create
laptop.sh script with content:

xrandr --output LVDS --mode 1024x768 --set BACKLIGHT_CONTROL legacy --output VGA --mode 1024x768
# This will restore backlight to 100%
xbacklight -set 100


I found it much safer/useful to control laptop display backlight,
instead of switching LVDS output off, because it is easier to switch
resolution back, to smaller (something is still visible on display) when
the external monitor is taken away by crazy administrator guy :)

References


Xorg RandR 1.2 Wiki

mergefb - dual monitor setup on a linux laptop

Sunday, April 19, 2009

Testing EJB out of application server - Part 3. Spring managed DAO tier

After a short delay and thinking about giving Spring to manage stateless DAO tier beans, here comes 3-rd part of the article. There were some issues that had to be investigated to make the topic complete. Most of all, there was an open question of controlling Hibernate Session
life cycle, to have OpenSessionInView pattern working.
Let's define what we want to gain with DAO bean management by Spring:
1. Keep DAO beans working with EntityManager to make CRUD operations with entities.
2. Make switch from EJB' @Stateless (and even @Stateful) annotations to Spring' @Service (or have beans configured in XML file).
3. Have bean auto wiring in DAO with Spring IoC, avoiding JNDI lookups and extra configuration in XML file.
4. Ensure transaction and session management is compliant with EJB approach to handle entity beans lazy initialization problem (here I refer to interceptor and callback/template approaches).

Let's examine what changes are to be done to our simple Internet shop application.

DAO tier


Since we do not need to manage DAO beans with EJB but Spring, just change annotations on DAO bean class level:

@Service("purchaseDAO")
public class PurchaseDAOImpl extends AbstractDAO<Purchase> implements PurchaseDAO {
public PurchaseDAOImpl() {
super(Purchase.class);
}
}


Here, we used @Service that is recognized by Spring component scanning functionality we already familiar with. Also, DAO interfaces do not need @Local nor @Remote annotation (so they are removed):

public interface PurchaseDAO extends DAO<Purchase> {
}


There is also a simplification in application context configuration file - DAO beans refering through JNDI to stateful beans managed by EJB have been removed.
Important configuration step is adding <context:annotation-config/> to application context configuration file. This will include support for JPA annotations like @PersistenceContext. To have everything working we need to define bean, with name of persistence unit, which refers to EntityManagerFactory:

<bean id="entityManagerFactory"
class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
<property name="targetObject"
ref="entityManagerBean"/>
<property name="targetMethod"
value="getEntityManagerFactory"/>
</bean>

<alias alias="ishopPU"
name="entityManagerFactory"/>


OpenSessionInView pattern


To be correct, in Spring terms pattern is named OpenEntityManagerInView. There is a significant difference between this new approach and old-style one used by applications, built upon Hibernate ORM, to keep Hibernate Session open for a request processing cycle.
Opening Hibernate Session in filter and binding it to SessionFactory these applications relied on the fact entity beans extracted in DAO tier are kept attached to this session until request has been processed and session is closed, so access to lazily initialized references would not cause an exception.
But the approach, in it's original implementation, does not work for EJB - each transactional method is completed within its own EntityManager instance which holds reference to it's distinct Hibernate Session; opening new session does not make any affect.
Instead, using TransactionSynchronizationManager, we bind EntityManager to EntityManagerFactory so each transactional method executed with the same EntityManager, and as result the same Session, instance used:

@Stateful(name = "openSessionInViewCallback")
@LocalBinding(jndiBinding = "openSessionInViewCallbackLocal")
public class OpenSessionInViewCallbackImpl implements OpenSessionInViewCallback {
@PersistenceContext(type = PersistenceContextType.EXTENDED)
private EntityManager entityManager;

@PersistenceUnit(unitName = "ishopPU")
private EntityManagerFactory entityManagerFactory;

@Init
public void invoke(Runnable r) {
TransactionSynchronizationManager.bindResource(entityManagerFactory, new EntityManagerHolder(entityManager));

r.run();
}

@Remove
public void remove() {
TransactionSynchronizationManager.unbindResource(entityManagerFactory);
EntityManagerFactoryUtils.closeEntityManager(entityManager);
}
}


In @Init method we bind entity manager, and unbind it on callback method completion with @Remove cleanup. For more information on Spring implementation of the pattern please refer to org.springframework.orm.jpa.support.OpenEntityManagerInViewFilter and org.springframework.orm.jpa.support.OpenEntityManagerInViewInterceptor

Source code


Can be downloaded here.

That's it!

Sunday, April 5, 2009

Testing EJB out of application server - Part 2. Integrating Spring

The aim


In Part 1 I mentioned Spring's transaction management API. I believe this is quite a serious reason to think about integrating Spring framework into EJB application. But not the only one. Spring provides easy way to maintain multiple application configurations with IoC making testing configuration of your application independent of production one. Other features, like JNDI support, integration with JMS, JMX, mail and other services make the framework so popular.
Returning to our internet shop application we have to decide what we want to leave managed by EJB and what we want to put under Spring's control, and look at other options we have.

Data access. We considered JPA as an appropriate way to configure entity and DAO beans that provide basic functionality to access former. DAO stateless beans usually do not depend upon any other application tier, and really seldom use each other; EntityManager gives all an ordinal DAO needs.

Business services. This is really a good place Spring may take care of. Injecting dependencies on other beans (e.g. DAOs), configuration parameters provided within application configuration files, managing transactions is where Spring really strong. Moreover, with Spring testing module we can use these features directly in test classes.

Web services. JSR-181 defines API that is available within stateless beans to define them as providing Web services methods. EJB3 greatly supports this API, and though there is an option to use XFire framework together with Spring, to configure Web services, I see no reason to keep away from standard. Sometimes it is better avoid getting deep into framework hell. Though not always :)

Sample application changes


Domain tier


It is not a surprise that there is nothing to be changed. Entity beans still use JPA for O/R mapping that is controlled by EJB container.

Data access tier


There is one thing that needs to be done to each DAO implementation bean. Since we are going to publish DAO stateless beans in JNDI, so they will be available to Spring, we will use @LocalBinding JBoss annotation to provide bean JNDI name for production environment, as follows:
@Stateless(name = "productDAO")
@LocalBinding(jndiBinding = "productDAOLocal")
public class ProductDAOImpl extends AbstractDAO<Product> implements ProductDAO {
// methods implementations
}

Every DAO bean is configured in application context configuration file like this one:
<jee:jndi-lookup id="productDAO"
proxy-interface="org.ishop.dao.ProductDAO"
jndi-name="productDAOLocal"/>

Unfortunatelly I do not know if there is any way to avoid this, like using automatic scanning.

Service tier


Minor changes should be done to service methods - to switch to Spring transaction API we change @TransactionAttribute to @Transactional and specify other options, if needed. Also dependency injection is now controlled by Spring, based on bean name or type wiring, or on @Autowired annotation:
@Service("productService")
public class ProductServiceImpl implements ProductService {
@Autowired
private ProductDAO productDAO;
@Autowired
private PurchaseDAO purchaseDAO;

@Transactional
public void save(Product product) {
productDAO.save(product);
}

@Transactional(readOnly = true)
public Product getProduct(Long id) {
return productDAO.get(id);
}

@Transactional
public Purchase purchase(Long productId, Date paymentDate) {
// method implementation
}
}

Please notice that we used @Service that allows us to skip configuring bean in application context configuration file, relying on auto-detection option:

We need to configure transaction management - Spring will use container JTA transaction manager to which it will redirect all transaction processing requests, based on own transaction annotations support and handling:
<bean class="org.springframework.transaction.aspectj.AnnotationTransactionAspect"
factory-method="aspectOf">
<property name="transactionManager"
ref="platformTransactionManager"/>
</bean>

<bean id="platformTransactionManager"
class="org.springframework.transaction.jta.JtaTransactionManager"/>

Here we configured aspect to support @Transactional annotations. One principal change is made to the build script - Java classes are processed by AspectJ compiler. Another option is use dynamic proxies that will do the same thing. One significant disadvantage of this approach is that to handle calls between business methods of the same service properly they should be made through the proxy.
One more good thing about Spring transaction management API is that service method may throw any exception (checked or unchecked). Instead of annotating exception class with @ApplicationException as EJB spec requires, in @Transactional annotation parameters we may specify which exception(s) should cause transaction rollback and which will lead to commit. Really nice.

Web services


As well as within business tier, Web service need just minor changes in API - replacement of @EJB with @Autowired and adding SpringContextInterceptor interceptor to the list of method call interceptors:
@Interceptors({SpringContextInterceptor.class, OpenSessionInViewInterceptor.class})
public class ProductWebServiceImpl implements ProductWebService {
@Autowired
private ProductService productService;
// methods skipped
}

SpringContextInterceptor is responsible for application context startup when application Web service accessed the first time.
public class SpringContextInterceptor extends SpringBeanAutowiringInterceptor {
private BeanFactory beanFactory;

protected BeanFactory getBeanFactory(Object o) {
if (beanFactory == null) {
beanFactory = new ClassPathXmlApplicationContext("/applicationContext.xml").getAutowireCapableBeanFactory();
}
return beanFactory;
}

@PreDestroy
@PrePassivate
public void releaseBean(InvocationContext invocationContext) {
super.releaseBean(invocationContext);
}

@PostConstruct
@PostActivate
public void autowireBean(InvocationContext invocationContext) {
super.autowireBean(invocationContext);
}
}


Changes to unit tests


Spring provides a module to support bean wiring into test classes for both jUnit and TestNG. What we need to change in AbstractPersistenceTest is set up application context and remove lookup() method since we do not rely on bean wiring with EJB:
@ContextConfiguration(locations = {"classpath:/test-applicationContext.xml"})
public abstract class AbstractPersistenceTest extends AbstractTestNGSpringContextTests {
@BeforeClass(groups = "database")
public final void setUpDatabase() throws NamingException, IOException {
Properties props = new Properties();
props.load(AbstractPersistenceTest.class.getResourceAsStream("/persistence.properties"));
new InitialContext(props);
}

@BeforeClass(dependsOnMethods = "setUpDatabase")
protected void springTestContextPrepareTestInstance() throws Exception {
super.springTestContextPrepareTestInstance();
}
}

Here method springTestContextPrepareTestInstance has been overriden to be called when EJB container has been set up by setUpDatabase().
Testing code does not need serious changes as well as production code does not - use dependency injection with @Autowired instead of looking up for required resource in JNDI.
I strongly believe Spring offers a lot of features which when used in enterprise level applications may reduce coupling of beans and tiers, making daily development and testing easier.

Source code


ishop.ejb-testng-spring.zip

Thanks you!