There was a null pointer issue that was tripping up enough people with 3.0.3 that I created a small 3.0.4 release to address the issue.
The issues fixed are:
with CORE-1423 being the one most people were running into.
You can download 3.0.4 from http://liquibase.org/download and through the maven repository system once it gets mirrored through.
Performance
The major focus of this release is performance improvements:
If you are still seeing any unexpected performance problems, file a bug report
You can download 3.0.5 from http://liquibase.org/download and through the maven repository system once it gets mirrored through. If you have any questions or problems, visit the forums or bug tracker
Notable Bugs
Potential Gotchas
Three bug fixes related to data type translations went into 3.0.5 which may cause issues for people that had been relying on incorrect behavior
Full Change Log
For the last two months I have been working full time as an employee with Datical, a startup that is building an enterprise-ready database change management solution on top Liquibase.
For me, it has been great because I am now able to devote considerably more time to Liquibase than I ever have before. Since starting Liquibase 7 years ago, I’ve ran it as a side hobby–spending a few night and weekend hours doing what I can to progress the project. However, as the community and user base has grown it had become difficult to keep pace. You may have noticed that during the 3.0 development timeline, I tended to oscillate between spending all my time writing code without answering any user questions and answering questions without writing code simply because I just didn’t have enough time to do everything I needed to.
So what does this change in employment mean for Liquibase? Nothing but good things!
Liquibase remains and always will be a freely available open source project. At Datical, we know the value of a strong open source project and community, and my primary responsibility at Datical is to spend as much time on Liquibase as possible in order to improve it for ourselves and for everyone else. In short, my job is to do what I have always done but with more focus and more resources.
Now that I have full days worth of time to spend on Liquibase, I have already been able to pick up the pace of Liquibase releases plus keep up with the user forums. It has been great!
In the coming months, look for more great changes to Liquibase including:
If you would like more information on Datical and DaticalDB you can read their official launch announcement. Or, as always, ask questions in the blog comments, in the Liquibase forum, or contact me directly at nathan.voxland@liquibase.org.
I’m happy to announce Liquibase 3.0.6 has been released. While the previous release focused on performance, this release was focused on catching up with pull requests and resolving some of the higher value bugs.
Some highlights and important notes:
Download Liquibase from http://liquibase.org/download or from the Maven repository as it winds its way through the mirror process. Visit the user forums if you have any questions.
All Closed Issues:
After far too long, a new version of the Liquibase-Hibernate extension has been released!
Downloads, documentation, and issue tracking is hosted at github.com/liquibase/liquibase-hibernate. The extension is also available through Maven with group org.liquibase.ext, artifacts liquibase-hibernate3 and liquibase-hibernate4.
New Features
Liquibase 3.0 Support
If you’ve been stuck on Liquibase 2.0 because of the old hibernate integration, the wait is over. The extension requires Liquibase 3.0.6+, so make sure you are on the newest version of Liquibase.
Hibernate 4 Support
There are two separate jar files available: liquibase-hibernate3.jar and liquibase-hibernate4.jar. Use the version that corresponds to your Hibernate version
EJB3 Configuration Support
If you are using the EJB3-style persistence.xml file, you can now use a database url of “hibernate:ejb3:yourPersistenceUnit“.
Spring Configuration Support
If your Hibernate configuration is specified in Spring, you can now use a database of “hibernate:spring:your/path/spring.xml?bean=sessionFactory” to pull the Hibernate configuration from your spring context.
Programmatic Configuration Support
You can now create your Hibernate configurations dynamically by implementing liquibase.ext.hibernate.customfactory.CustomClassicConfigurationFactory or liquibase.ext.hibernate.customfactory.CustomEjb3ConfigurationFactory and reference the classes in your database url. For example, “hibernate:classic:com.example.YourFactory” or ”hibernate:ejb3:com.example.YourFactory”
Upgrade Notes:
There has been a lot of activity the last couple weeks on the Liquibase extensions I manage and/or have source access to. There were a few running up against the Oct 15th jira-svn-shutdown deadline and also several that had not been updated to support Liquibase 3.0 yet.
For all of the extensions, I have moved their source repository out of the shared liquibase.jira.com subversion repository and into their own repositories at https://github.com/liquibase. They all now have their own issue tracking, wiki, and releases tab in their Github projects, which is the pattern I will be using for all the Liquibase-managed extensions.
I also released the new builds into Maven under the org.liquibase.ext group (some may still be propagating through the mirrors).
The recently updated extensions are:
Oracle Database Support (https://github.com/liquibase/liquibase-oracle)
Improved support for Oracle database. Adds additional refactoring tags and snapshot support for Oracle-specific object types
Hana Database Support (https://github.com/liquibase/liquibase-hanadb)
Support for SAP’s Hana Database
Teradata Database Support (https://github.com/liquibase/liquibase-teradata)
Adds support for Teradata database. Still in beta because I am looking for Teradata users to test it
No Change Log Update (https://github.com/liquibase/liquibase-nochangelogupdate)
Makes Liquibase run without updating the DATABASECHANGELOG table. Not what you normally want, but there are times it is handy
java.util.Logger Support (https://github.com/liquibase/liquibase-javalogger)
Log through java.util.Logger rather than STDERR
Sequence Table (https://github.com/liquibase/liquibase-sequencetable)
Support “sequence tables” like Hibernate can use for databases that do not support native schemas
Modify Column (https://github.com/liquibase/liquibase-modify-column)
Support for the deprecated <modifyColumn> change from Liquibase 1.x
The main “Liquibase Extension Portal” is still at liquibase.org/extensions, the above extensions are simply the subset of extensions that I tend to manage. In the upcoming weeks, I will be working to improve the extension portal and the extension writing process in general (including better documentation)
Lots of good bugfixes in 3.0.7, but major improvements are:
Discovered Checksum Issue: It was recently discovered that a change in the 3.0.5 release caused changeSets with <update> and <delete> changeSets with <where> clauses to compute a different checksum version than the 3.0.0-3.0.4 release. I decided to leave the 3.0.5 fix in because at this point, people would be affected by either leaving or reverting the change and keeping it in will preserve the bugfix. If you are upgrading from 2.x or 3.0.5+, this will not affect you. If you are upgrading from 3.0.0 through 3.0.4 and have no <update> or <delete> changeSets, this will not affect you. If you do run into this, you simply have to null out the md5sum column in databasechangelog for the affected changesets. This can be done either through an “update databasechangelog set md5sum=null” call or a more targeted ”update databasechangelog set md5sum=null where id=’AFFECTED_ID’ and author=’AFFECTED AUTHOR’”. There is also a “liquibasec learCheckSums” command that runs the update query.
Download
Download Liquibase from http://liquibase.org/download or from the Maven repository as it winds its way through the mirror process. Visit the user forums if you have any questions.
All Closed Issues
One problem with running an open source project is that we don’t really have “customers” to contact for feedback and product suggestions. I can definitely just add features that I think make sense, but one thing I’ve learned is that people use Liquibase in amazing ways I would never have imagined.
That is where the Liquibase survey comes in: it is your chance to provide feedback on how you use Liquibase and what is important to you so that I can better prioritize efforts over the next year.
Your participation and time is greatly appreciated.
If you know others that use Liquibase, please forward them the survey as well.
Even better, if you know someone who is NOT using Liquibase, send them the survey as well.
The more information I can gather the better picture I will have.
If you haven’t yet filled out the Liquibase 2013 Survey now is the time–it will be available only through the end of Friday, November 15th.
There have been some great comments and feedback so far and thank you to everyone who has taken the time so far to submit a response.
For those interested, I will post a summary of the results before the end of November.
The Liquibase hibernate extension has been updated for both hibernate 3 and hibernate 4.
This release adds support for catching changes to sequences and unique constraints as well as a few bugs around data type handling.
See https://github.com/liquibase/liquibase-hibernate for more information and to download the 3.1 release.
UPDATE: A null pointer issue was found in the 3.1 release and fixed in a now available 3.2 release.
A new Liquibase-Redshift extension is now available at https://github.com/liquibase/liquibase-redshift. If you are interested in trying out Liquibase with Amazon Redshift, give it a try and let me know how well it works.
You can download the most recent builds from the “releases” tab.
Liquibase 3.0.8 is officially released. It is purely a bugfix release that covers improvements to data type handing, performance improvements, and much more.
Download
Download Liquibase from http://liquibase.org/download or from the Maven repository as it winds its way through the mirror process. Visit the user forums if you have any questions.
All Closed Issues
Liquibase 3.1.0 has been released. For most people 3.1 is a drop in replacement for 3.0.
If you use <includeAll> with relative paths or are running against MaxDB, DB2 for iSeries, or InterSystems Cache make sure you read the upgrade guide.
Besides bugfixes, 3.1 adds the following major features:
Offline Database Support!
One of the most requested features is the ability to use Liquibase when you do not have direct access to the database. We have always had updateSql support, but it still required a database connection to know what has been ran and what type of database to generate SQL for.
With the new “offline” support, you can now define a database connection URL like “offline:mysql” or “offline:oracle?version=11.1” when you run updateSql.
In offline mode, the change history is tracked in a local csv file which can be controlled with the “changeLogFile” URL parameter. For example: offline:mysql?changeLogFile:/home/db/changelog.csv
For more information, see the offline database documentation
Alternate Changelog History Tracking
With 3.1, you can now create extensions that manage changelog history tracking differently than the standard “DATABASECHANGELOG” table. For example, https://github.com/liquibase/liquibase-filechangelog allows you to manage your changelog history using a local file rather than a table in the database.
New and Improved Change tags:
New and Improved Precondition tags:
Database Support Improvements
Extension/API Improvements
Other Features and Improvements
Links
Liquibase 3.1.1 has been released with a few bug fixes.
The most important fixes reverts how <createProcedure> checksums are computed so they are compatible with 3.0 and fix the new <rowCount> and <tableIsEmpty> preconditions.
The newest version can be download from liquibase.org/download
The Liquibase hibernate extension has been updated for both hibernate 3 and hibernate 4.
Major changes include:
See https://github.com/liquibase/liquibase-hibernate for more information and to download the 3.3 release.
The new versions are also working their way through the maven mirror system
I’ll be giving a talk at Percona Live in Santa Clara on April 3rd titled “Liquibase: Source Control For Your Schema“.
It is a beginner-level course covering Liquibase basics plus some best practices. More information
If you will be there, stop by and say hi. It’s always great to hear from people using or considering using Liquibase. If you have not yet registered, use the promotional code “SeeMeSpeak” for a 10% discount.
I posted the slides from my Percona Live talk at https://www.percona.com/live/mysql-conference-2014/sessions/liquibase-source-control-your-schema.
I modified them from the actual talk slides to also include my speaking notes so they will be more helpful for those who were not able to make it to the talk or if you wish you took better notes.
I’m hoping to post a recorded version of the talk in the next few weeks as well.
Liquibase 3.2.0 has been released. As usual, it can be downloaded from the Liquibase download page and is available in the Maven repository as org.liquibase/liquibase-core.
For most users, upgrading will be seamless but if you have written Liquibase extensions there have been some API changes. See http://www.liquibase.org/v3_2_upgrade.html for details.
The following extensions have been updated to work with 3.2.0 and/or include bugfixes:
And/Or/Not logic in context expressions
Context attributes can now include complex expressions such as “qa or (production and master)”. See the context documentation for more information
Improved JSON and YAML changelog parsing
The changelog parsing logic has been greatly refactored to provide full feature parity between the XML, JSON, and YAML parsers.
New Command: Snapshot
The command line interface supports a new “snapshot” command that will output a report of the objects Liquibase sees in a database. This is the model that would be fed into diff/diffChangeLog operations but doesn’t perform any comparison logic, is simply reports on what it sees.
Liquibase SDK
The Liquibase SDK will provide utilities and features not needed for standard Liquibase usage such as testing tools and extension writing help. The 3.2.0 release provides the first pieces of the SDK:
For more information on the SDK, see the SDK Documentation.
Improved Performance
Various improvements in memory usage and optimizations in database interactions
DatabaseChangeLog SQL available with Offline Database
New “outputLiquibaseSql” flag allows inclusion of DatabaseChangeLog table create/insert statements with updateSql command
And Much More:
The Quick Start Guide works well for starting Liquibase on a new project because your empty changelog file matches your empty database. However, when you have an existing project with an existing database things are more complicated.
Unfortunately, there is no simple “this is how you do it” answer because there is so much variations in projects, processes and requirements. Liquibase provides many tools to help the process, but it is up to you to decide the best way to combine them for your particular situation.
When adding Liquibase to an existing project there are basically two approaches: “Make it look like you’ve always been using Liquibase” and “Just start using Liquibase”
The goal of this approach is to have a changelog file that matches the current state of your database. You can run this changeLog against a blank database and the final result will be indistinguishable from your existing databases–as if you used Liquibase from the beginning. This approach is usually the best long term, but it can be more work up front.
Creating the changelog to match your database can be done automatically using the generateChangeLog command or be done manually. For any database larger than a few tables, the generateChangeLog command is usually a good idea but make sure you go through the generated changeSets to ensure they are correct. Liquibase does not always detect more complex structures like stored procedures or details like if an index is not clustered. Also, ensure data types are as you expected them.
Once you have your changeLog, you need a way to ensure that the pre-Liquibase changeSets are only ran on new, empty databases. The easiest way to do this is generally to use the changeLogSync or changeLogSyncSQL command to execute (or generate) the SQL that marks the starting changeSets as already ran without actually executing them.
As an alternative to the changeLogSync command, you can add contexts on the pre-Liquibase changeSets such as <changeSet ... context="legacy"> and when you run Liquibase on a new database you run with liquibase --contexts=legacy update and on an existing database you run with liquibase --contexts=non-legacy.
Finally, you can add <precondition onFail="MARK_RAN"> tags to the generated changeSets. For example, if you have a <createTable tableName="person"> changeSet, you would add <preconditions onFail="MARK_RAN"><not><tableExists tableName="person"/></not></preconditions> tag. Adding preconditions requires more changes to the changeLog file and introduces a performance penalty because Liquibase must check the database metadata for each changeSet the first run through, this approach is usually best used in isolated cases only.
Often times a part of the reason to move to Liquibase is because your schemas have diverged over time, so an important question to answer is “If I’m making the changelog file match the current state, what is the current state?” Usually the best answer to that question is “the production database” but it can vary.
How divergent your schemas are will also affect which of the above techniques you use to populate the DatabaseChangeLog table, and it will often times make sense to use multiple approaches. For example, you may want to generate your base changeLogs from the production database and use changeLogSyncSQL to be able to mark them ran on everything from production down. Then you can add your non-released changeSets to the changeLog file with a precondition checking if it has already ran. That will allow Liquibase to automatically figure out the correct state for all your databases from development through production.
Instead of building up a changeLog to match your existing database, you can instead just declare “from now on we are using Liquibase”. The advantage to this is that it much easier to set up because it is just a mandate. Usually this works best going from one version to the next because your databases are all in a reasonably consistent state and you simply start tracking database changes in your next version using Liquibase. Because Liquibase only looks at the DatabaseChangeLog table to determine what needs to run, it doesn’t care what else might be in your database and so it will leave all your existing tables alone and just run the new changeSets.
The biggest disadvantage to this approach is that you cannot bootstrap an empty database with Liquibase alone. A work-around is to take a pre-Liquibase snapshot using your database backup tool and use that as your database seed. Any time you need to create a new database, you first load in the seed and then run Liquibase update.
Depending on how much variation you have between your schemas, even with this approach you may need to rely on preconditions or a “mark changes ran” script in order to standardize and handle those variations.
Finally, remember that starting to use Liquibase–especially on an existing project–isn’t just about how you bootstrap your changeLog file. It is also a question of how you introduce Liquibase into your existing processes and culture.
For many companies and projects, everyone realizes the problems that need fixing and are on board with the advantages of change. For others, however, there can be entrenched interests and strong resistance similar to any other process change. Liquibase provides many tools and approaches that can be used to ease it into an existing process such as SQL output, SQL formatted changelogs, diffChangeLog and more that can be combined in ways that works best for your group.
If you know that introducing Liquibase is going to be complex, either from a technical or processes standpoint, it is usually best to introduce it slowly. Start with it on a new project as a trial run and once you have a good grasp of how it works and available options, apply it to other existing projects.
NOTE: This post has been incorporated into the documentation at http://www.liquibase.org/documentation/existing_project.html
Liquibase 3.2.1 has been released. As usual, it can be downloaded from the Liquibase download page and is available in the Maven repository as org.liquibase/liquibase-core.
3.2.1 is purely a bugfix release. The only potential update issue could be a fix for incorrect checksums introduced in 3.2.0. If you used defaultValueNumeric, defaultValue or used createProcedure or sql tags you may have seen unexpectedly changed checksums with 3.2.0. With 3.2.1, those have been reverted back to the correct 3.1.x version. If you are updating from 3.1.x it will be a smoother update from 3.2.0. If you already updated to 3.2.1 and manually fixed your checksums in the databasechangelog table, they will need to be updated again.
Full change log:
UPDATE: There is still a couple checksum issues updating from 3.1.1 to 3.2.1. I’ll get those fixed up in a 3.2.2 release today or tomorrow.