Rakefile shall launch unit tests for only changed modules

Bug #1537063 reported by Bogdan Dobrelya
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
High
Maksim Malchuk
8.0.x
Fix Committed
High
Maksim Malchuk

Bug Description

Openstack infra gate-fuel-library-puppet gates call "bundle exec rake spec" for each submitted patch.
While the Rakefile in the fuel-library repo launches unit tests for all modules, which is not optimal and only consumes more resources and time.

It shall instead repeat the logic of the fuel_unit_tests script https://github.com/openstack/fuel-library/blob/master/utils/jenkins/fuel_unit_tests.sh#L83-L84 and run only unit tests for modules changed by the patch under test

Changed in fuel:
importance: Undecided → High
tags: added: non-release
Changed in fuel:
milestone: none → 9.0
tags: added: area-library
description: updated
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

Looks like not a way to go. "If we change something like galera that is used by openstack but we don't check openstack how will we know if there are regressions"?

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

There is another opinion though, that the galera module changes shall be only covered by the unit tests of the *galera* module but not openstack or osnailyfactor. Those shall be covered by integration tests, like Noop framework. So, this point makes the bug valid back :) Let's discuss it in mail list perhaps

no longer affects: fuel/mitaka
Changed in fuel:
status: Confirmed → Opinion
Revision history for this message
Aleksandr Didenko (adidenko) wrote :

"If we change something like galera that is used by openstack but we don't check openstack how will we know if there are regressions"

I suppose that such problems are not a subject for unit tests. If I change something in galera module, then the only goal of unit tests is to check that functionality of puppet classes/function/resources of galera module is not broken, that's it. Any composition/integration layer issues (like the one described above) will be caught by noop tests.

Revision history for this message
Alex Schultz (alex-schultz) wrote :

I personally would like to keep some sort of integration rake tests that isn't NOOP for fuel-library because NOOP 1) takes forever and 2) does not actually run cleanly on my OSX machine (too many linux assumptions, not enough mocks). I think it would be acceptable to make the unit tests only check the changed modules (as done via fuellib_unit_tests) but I think it's also a good idea to run all of the modules in at least 1 job to check integration via rake rather than noop.

Changed in fuel:
status: Opinion → Confirmed
Revision history for this message
Maksim Malchuk (mmalchuk) wrote :

Related fix proposed to branch: master
Review: https://review.openstack.org/272346

Changed in fuel:
status: Confirmed → Fix Committed
Changed in fuel:
status: Fix Committed → In Progress
Dmitry Pyzhov (dpyzhov)
tags: added: tech-debt
Revision history for this message
Matthew Mosesohn (raytrac3r) wrote :

We should consider skipping 8.0 for this and fix just in master.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (master)

Fix proposed to branch: master
Review: https://review.openstack.org/274286

Revision history for this message
Maksim Malchuk (mmalchuk) wrote :

This is actually not a bug, but CI enhancements. The changes don't included in the release and only intended to speed up the development process.

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Related fix proposed to fuel-infra/jenkins-jobs (master)

Related fix proposed to branch: master
Change author: Maksim Malchuk <email address hidden>
Review: https://review.fuel-infra.org/16576

Changed in fuel:
assignee: Maksim Malchuk (mmalchuk) → Dmitry Kaigarodеsev (dkaiharodsev)
Changed in fuel:
assignee: Dmitry Kaigarodеsev (dkaiharodsev) → Maksim Malchuk (mmalchuk)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/274286
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=f60d5ed950baefd2223388d66e71b7f037affffd
Submitter: Jenkins
Branch: master

commit f60d5ed950baefd2223388d66e71b7f037affffd
Author: Maksim Malchuk <email address hidden>
Date: Sat Jan 30 00:26:08 2016 +0300

    Rakefile support for Git changes

    This commit enables integration with Git. Now before run
    RSpec or Lint tasks Rakefile will check for Git changes
    and run these tasks on changed modules only otherwise
    they ran against all modules.

    Change-Id: I79ffbb8e2ed0c10cf03a1b2e176b75b314a3e79d
    Closes-Bug: #1537063
    Closes-Bug: #1513900

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (stable/8.0)

Fix proposed to branch: stable/8.0
Review: https://review.openstack.org/274769

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on fuel-library (stable/8.0)

Change abandoned by Maksim Malchuk (<email address hidden>) on branch: stable/8.0
Review: https://review.openstack.org/274769
Reason: https://review.openstack.org/#/c/274712
should be merged first

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (master)

Fix proposed to branch: master
Review: https://review.openstack.org/274878

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (stable/8.0)

Reviewed: https://review.openstack.org/274769
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=4734ab2aec82fa3ccb1aa7a4143f9c3fe11fddaa
Submitter: Jenkins
Branch: stable/8.0

commit 4734ab2aec82fa3ccb1aa7a4143f9c3fe11fddaa
Author: Maksim Malchuk <email address hidden>
Date: Sat Jan 30 00:26:08 2016 +0300

    Rakefile support for Git changes

    This commit enables integration with Git. Now before run
    RSpec or Lint tasks Rakefile will check for Git changes
    and run these tasks on changed modules only otherwise
    they ran against all modules.

    Change-Id: I79ffbb8e2ed0c10cf03a1b2e176b75b314a3e79d
    (cherry picked from commit f60d5ed950baefd2223388d66e71b7f037affffd)
    Closes-Bug: #1537063
    Related-Bug: #1513900

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/274878
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=7de0ee245c03736e22c5b6943e65a7d20deeb151
Submitter: Jenkins
Branch: master

commit 7de0ee245c03736e22c5b6943e65a7d20deeb151
Author: Maksim Malchuk <email address hidden>
Date: Mon Feb 1 21:38:35 2016 +0300

    Parameter skip_file didn't work with git changes

    This commit fixes issue when skip_file didn't used at all

    Change-Id: Iaa783fb37d8000f0578019c8e257ab8e4c992e9b
    Partial-Bug: #1537063

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (stable/8.0)

Fix proposed to branch: stable/8.0
Review: https://review.openstack.org/274944

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (stable/8.0)

Reviewed: https://review.openstack.org/274944
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=c4c4c9943ead8ea573ee9ba6e051be57576316cc
Submitter: Jenkins
Branch: stable/8.0

commit c4c4c9943ead8ea573ee9ba6e051be57576316cc
Author: Maksim Malchuk <email address hidden>
Date: Mon Feb 1 21:38:35 2016 +0300

    Parameter skip_file didn't work with git changes

    This commit fixes issue when skip_file didn't used at all

    Change-Id: Iaa783fb37d8000f0578019c8e257ab8e4c992e9b
    (cherry picked from commit 7de0ee245c03736e22c5b6943e65a7d20deeb151)
    Partial-Bug: #1537063

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Related fix merged to fuel-infra/jenkins-jobs (master)

Reviewed: https://review.fuel-infra.org/16576
Submitter: Aleksandra Fedorova <email address hidden>
Branch: master

Commit: 513c7a12a0481b4155a8442f7abe17681be3538e
Author: Maksim Malchuk <email address hidden>
Date: Sat Jan 30 21:32:55 2016

Remove fuellib review_syntax_check and unit_tests jobs

Jobs fuellib_review_syntax_check and fuellib_unit_tests
should be removed from stable/8.0 and master because we
already have the same functionality in the upstream
OpenStack jobs.

Change-Id: Ic7ce913a410d1e6379abf02a4eeb9cb197f85ba3
Closes-Bug: #1513900
Related-Bug: #1537063

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.