Data Fabric Release Notes

Owned by Youen Chene (Unlicensed)
Last updated: Jun 05, 2018 by Stéphanie Brion (Unlicensed)
6 min read

Version 1.19 - June 2018

Features :

  • Data Fabric : be able to choose a runtime constraint when you create or upgrade a job (extra constraint can be GPU, HPC, EDGE, depends on the material available on your platform)


From 1.18.20180527131007-326.2 - June 2018

Features :

Saagie Platform 2.0 beta


Version 1.17 & 1.18 - March 2018

Data Governance versions : Data Governance Release Notes

Features : 

  • Update of Spark capsule :
    • from 1.6.1 to 1.6.3
    • from 2.1.0 to 2.1.2
    • new version 2.2.1
    • new version 2.3.0
    • version 2.1.2 now recommended


Version 1.16 - March 2018

Features : 

  • Add Intercom chat box in datafabric
  • Security updates


Version 1.15 - March 2018

Features : 

  • Adding RStudio as a new DataScience notebook
  • Security updates


Version 1.14 - February 2018

Features : 

  • Adding Apache Zeppelin as a new DataScience notebook 


Version 1.13 - February 2018

Features :

  • Data Governance becomes Datalake service 


Capsule Upgrade - Python Jupyter Notebook with Saagie Plugin

Features :

  • Saagie Plugin within Python Jupyter Notebook for continuous delivery from notebook to python job.


Version 1.12 - February 2018

Minor version : migrate and rotate log of nginx, traefik, haproxy, mongo, mysql, nexus,pgsql, elasticsearch


Version 1.11 - February 2018

Minor version : a v1 API composed of data fabric, data governance and ressources on a single endpoint (documentation coming soon)


Version 1.10 - January 2018

Minor version : first step towards deployment on Kubernetes, it works on future layout (unification of Saagie UIs)


Version 1.9 - January 2018

Marketing & Sales version: Release Note Saagie Data Fabric 1.9.pdf

Features:

  • Jupyter Notebook 2.4.1 => 5.2.1
    • Python Notebook: upgrade to Jupyter scipy notebook 
    • R Notebook: upgrade to Jupyter R notebook 
    • Ruby Notebook: upgrade from Ruby v2.3 to Ruby v2.4 
    • Julia Notebook: upgrade from Julia v0.6.0 to v0.6.2 
    • Haskell Notebook: upgrade to Haskell v8.0.2



Version 1.8 - January 2018

Features:

  • API: New route to get a user logged to the API, which returns a token for the session instead of redirecting to the platform list.
  • API: New route to get permissions and roles of the user, by giving his session token.


Version 1.7 - December 2017

Marketing & Sales version: Release Note Saagie Data Fabric 1.7

Features:

  • Expose internal URL on your Docker jobs in order to deploy micro services architecture within Saagie


Version 1.6 - December 2017

Marketing & Sales version: Release Note Saagie Data Fabric 1.6

Capsules:

  • HDFS: Upgrade version to 2.6.0
  • Hive: Upgrade version to 1.1.0
  • Impala: Upgrade version to 2.9.0
  • Sentry: Upgrade version to 1.5.1

Features:

  • Use the same users in Data Fabric and Hadoop services (Hive, Impala, Hue and HDFS)
  • Manage your users and groups in Data Fabric
  • User must set up his password on first connexion
  • Reset your password


Version 1.5 - September 2017


Marketing & Sales version: Release Note Saagie Data Fabric 1.5

Capsules:

  • Datamart: PostgreSQL
  • Datamart: ElasticSearch

Features:

  • Tag important job versions
  • Extend job search with status filter
  • Link to the public wiki in the page footer
  • Apps automatically access to Hadoop configuration XML files
  • Security fixes


Version 1.4 - May 2017

Marketing & Sales version: Release Note Saagie Data Fabric 1.4

Capsules:

  • HDFS: Transparent Encryption Option
  • HDFS: Encryption KeyStore can be in your own server

Version 1.3  - March 2017

Marketing & Sales version : Release Note Saagie Data Fabric 1.3

Capsules:

  • Python : versions 3.5.2 and 2.7.13.

  • Apache Spark : Python available on versions 3.5.2 and 2.7.13.

  • Drill : upgraded to 1.9

  • Datascience notebook : Jupyter Python/Spark 2.1.0

Features:

  • Saagie Manager becomes Saagie Data Fabric

  • Sqoop : Support big decimal format

  • Option to send an email at the end of a job whatever the status

PDF Full version :


Version 1.2 - January 2017

Capsules:

  • Apache Kafka : version 0.10.1.0 (option, please contact us).
  • Apache Spark : 2.1.0 available.
  • Apache Spark : 2.0.0 upgraded to 2.0.2.

Features:

  • Jobs : add a description to a job.
  • Jobs : add a comment to a job version.

Version 1.1 - December 2016

Capsules:

  • Docker: Batch mode for extraction processing.
  • Docker: private registry, be able to set url, versionning, option to de-activate manager authentification.

Features:

  • Settings: be able to change RAM, CPU and DISK.
  • Scheduling: remove retry on failure.
  • Jobs: stop/restart capabilities.
  • Jobs: be able to kill a job in requested status.

Fixed Bugs:

  • [CDP-201] - Saagie Manager - Create job with spaces in file name
  • [CDP-108] - Saagie Manager - Restart an error job automatically

Version 1.0 - Long Term Support - September 2016

Capsules:

  • Datalake: HDFS - High Availability,
  • Spark : 2.0 (available as beta).

Features:

  • Data pipeline,
  • Global Search.

Fixed Bugs:

  • [CDP-202] - Saagie Manager - Visibility issue in Platform history
  • [CDP-196] - Saagie Manager - Impossible to add a job with package filename including space


History

Until 1.0 each version of Saagie Manager is named regarding a space exploration milestone. 1.0 is of course apollo 11.

Apollo 11 was the first spaceflight that landed humans on the Moon. Mission commander Neil Armstrong and pilot Buzz Aldrin landed the lunar module Eagle on July 20, 1969, at 20:18 UTC.

Version 0.10 - Luna-9 - August 2016

Capsules:

  • Datalake: Impala - upgrade to version 2.5,
  • DataLake: Drill 1.7,
  • Datamart: MySQL 5.7.

Features:

  • New "Revolution" UI,
  • Create a job from another,
  • New panel to display capsules connection details,
  • Set resources (CPU, RAM, DISK) for all kind of jobs.

Fixed Bugs:

  • [CDP-163] - Re-Schedule a failed job
  • [CDP-173] - Platform order display
  • [CDP-177] - Reload platforms in dropdown after user added.
  • [CDP-184] - Manager GUI - Varenv display - Text overlap


History

Luna 9, internal designation Ye-6 No.13, was an unmanned space mission of the Soviet Union's Luna programme. On 3 February 1966 the Luna 9 spacecraft became the first spacecraft to achieve a soft landing on the Moon, or any planetary body other than Earth, and to transmit photographic data to Earth from the surface of another planetary body.

Version 0.9 - Astérix

Capsules:

  • Impala: upgrade to version 2.30,
  • Spark: option to use 1.6 available,
  • Saagie Present : alpha version available,
  • DataScience Notebook : upgrade to Jupyter 4.1 + one dedicated notebook by language (R, Python, Spark, Julia, Haskell) full of the best librairies.

Features:

  • Job's versioning for Sqoop, Spark, Java/Scala and Talend,
  • Configuration Management : environment properties,
  • Jobs now sorted by status and dates.

Fixed Bugs:

  • [CDP-98] - On disconnect got a json
  • [CDP-122] - Color for job killed status
  • [CDP-148] - Incorrect Start date on spark jobs
  • [CDP-152] - Issue on spark-ui access


History

Astérix, the first French satellite, was launched on November 26, 1965 by a Diamant A rocket from Hammaguir, Algeria. With Astérix, France became the sixth country to have an artificial satellite in orbit.

Version 0.8 - Voskhod 2

Capsules :

  • Spark : streaming/long job option,
  • Spark : access to spark UI when a spark job is pending,
  • Hadoop : ACL managed with Sentry.

Features :

  • new manager UI feature as search and pagination,
  • assets/jobs copy system between platform instance.


History

Voskhod 2 was a Soviet manned space mission in March 1965. The Vostok-based Voskhod 3KD spacecraft with two crew members on board, Pavel Belyayev and Alexey Leonov, was equipped with an inflatable airlock. It established another milestone in space exploration when Alexey Leonov became the first person to leave the spacecraft in a specialized spacesuit to conduct a 12 minute "spacewalk".

Version 0.7 - San Marco

Capsules :

  • Docker : Private Image
  • Java : Java 8 compatibility

Features :

  • handle big logs (download now possible),
  • email alerting on failed job,
  • auditing (hdfs, hive, impala),
  • LDAP with SSL authentication on impala, hive, hue.

Fixed Bugs :

  • [CDP-89] - Color issue on pending status
  • [CDP-90] - Remove button HS
  • [CDP-91] - HDFS : default replication settings
  • [CDP-100] - Jupyter Capsule Jupyter - Issue in deploiement
  • [CDP-110] - Mono request only on Hue


History

San Marco 1, also known as San Marco A, was the first Italian satellite, and one of the earliest non-Soviet/US spacecraft. Built in-house by the Italian Space Research Commission (Italian: Commissione per le Ricerche Spaziali, CRS) on behalf of the National Research Council, it was the first of five as part of the Italian-US San Marco programme.

Version 0.6 - Valentina

Capsules :

  • Docker (beta)
  • Jupyter (beta)

Features :

  • more readable execution time,
  • watch log during pending process,
  • display command line information in settings.

Fixed Bugs :

  • [CDP-48] - Job Browser empty despite of hive processing
  • [CDP-53] - Yarn Config - Job distribution issue


History

Valentina Vladimirovna Tereshkova (Russian: Валенти́на Влади́мировна Терешко́ born 6 March 1937) is the first woman to have flown in space, having been selected from more than four hundred applicants and five finalists to pilot Vostok 6 on 16 June 1963.

Version 0.5 - Mariner-2

Capsules :

  • R

Features :

  • show version information in connection details,
  • change Job Settings,
  • manual Job Launch (without scheduling),
  • platform manager user.

Fixed Bugs :

  • [CDP-75] - File upload progress bar


History

Mariner 2 (Mariner-Venus 1962), an American space probe to Venus, was the first robotic space probe to conduct a successful planetary encounter.

Version 0.4.5 - Alouette 1

Capsules :

  • Spark

Features :

  • see status & log history of a job,
  • delete a job,
  • change settings of a job.


History

Launched in 1962, Alouette 1 was Canada's first satellite, and the first satellite constructed by a country other than the USSR or the United States.

Version 0.4 - Youri

Capsules :

  • HDFS
  • Impala
  • Hive
  • SQOOP
  • Java-Scala
  • Talend
  • MongoDB
  • OpenData

Features :

  • batch scheduling,
  • watch job status & logs,
  • see connection details,
  • Hue integration.

History

Youri Gagarin was the first human in a spaceflight (USSR).