You are browsing the archive for 2013 October.

Organization Migration Complete

Ross Jones - 11/10/2013 in Community Updates, News

IMPORTANT: if you need help please see the forum at help.datahub.io.

In particular: you now need to be part of an organization to create datasets and to register an organization requires admin assistance (via the forum)

The planned organization migration has happened, and we’ve managed to get rid of a lot of spam, but not all.

As a result of the migration, all datasets must now belong to an organisation, and this has meant that datasets that were not part of a group previously, have been moved into a single org (Global – http://datahub.io/organization/global).

If you find that your datasets have been moved into Global, chances are you won’t currently be able to move them to another Organization without help. I’m planning to make it possible for users who created a dataset to be able to move it, but you will still need help in setting up an organization. If may be that you’re happy to leave it in the Global org, in which case you’re probably just waiting for me to enable you as an editor on the dataset – this will hopefully be before the end of the coming weekend, if you need it sooner, please get in touch and let us know.

We’re expecting that it is now a LOT harder for spammers to post to datahub.io, and only slightly harder for users, although there is still some cleaning up to do, and old data to purge, we’re pretty close now to a place where we can add new features (add some more ideas in the issue tracker) and improve the performance.

Whilst waiting for my migration script to finish last night (it took about 5 hours to clean spam and move the datasets) I also installed varnish, so if you see any caching irregularities, please just shout.

Organizations Upgrade

Rufus Pollock - 08/10/2013 in Community Updates, News

A quick note about an upcoming upgrade to help us address spam and improve the managability of datahub.io.

Anybody can currently post a dataset or create a group on datahub.io, and we think this is a good thing. However, it means that spammers can also post datasets, and they have. Whilst we certainly don’t want to close datahub.io from contributions, we do need to make changes to dramatically reduce the amount of spam being posted and we think that the we may have a way to achieve that.

CKAN 2.0 introduced idea of “Organizations” that own/publish datasets and which can have members who can be administrators (who can add users and manage the organization) and editors (who can only add datasets to the organization). This brings many features, not the least of which are that:

  • It provides a much richer permissions and authorization structure (based around the organization) that gives users greater control over who can, or cannot, edit and add datasets
  • It provides a clear organization-oriented structure for presenting and finding datasets
  • It will help us address spam problem by providing more control over who adds datasets as it will be a requirement that datasets are added to organizations.

We plan to enable organizations in the next few days. This will have several major effects:

  • You will only be able to create a dataset if you belong to an organization (and creating an organization at present requires approval from an Administrator)
  • Groups will be automatically migrated to organizations and the user-account that created them will be made the administrator. If you find you don’t have as much control over your organization as you thought, please let us know!
  • If you had datasets on datahub.io that were not part of a group, they will be added to a ‘Global’ organization and we will help you move them to a new organization should you wish to move them.

The migration is likely to take about a couple of hours, and so during this time datahub.io is likely to be rather unresponsive. As a result it is likely that we will take the site down for a short period of time but I’ll make sure that we notify the list should it look like it is going to take longer.

Thanks for your patience whilst we sort out the spammers, hopefully we’ll be seeing a lot less of them in the near future.